-- Logs begin at Mon 2022-08-01 07:01:55 UTC, end at Mon 2022-08-01 08:25:05 UTC. -- Aug 01 07:40:31.214881 dv-cinder-740146-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 07:40:31.223687 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 07:40:31 2022] *** Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: nodename: dv-cinder-740146-16-iscsi Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: machine: x86_64 Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: clock source: unix Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: pcre jit disabled Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: detected number of CPU cores: 4 Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: current working directory: / Aug 01 07:40:31.225652 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: detected binary path: /usr/bin/uwsgi-core Aug 01 07:40:31.226377 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: your processes number limit is 47680 Aug 01 07:40:31.226377 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: your memory page size is 4096 bytes Aug 01 07:40:31.226377 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: detected max file descriptor number: 2048 Aug 01 07:40:31.226377 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: lock engine: pthread robust mutexes Aug 01 07:40:31.226377 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: thunder lock: enabled Aug 01 07:40:31.226377 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 07:40:31.226377 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 07:40:31.265820 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: Python main interpreter initialized at 0x55a8d331f9c0 Aug 01 07:40:31.265820 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: python threads support enabled Aug 01 07:40:31.265820 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: your server socket listen backlog is limited to 100 connections Aug 01 07:40:31.265820 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: your mercy for graceful operations on workers is 90 seconds Aug 01 07:40:31.266156 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 07:40:31.266243 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: *** Operational MODE: preforking *** Aug 01 07:40:31.266499 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: *** uWSGI is running in multiple interpreter mode *** Aug 01 07:40:31.266499 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: spawned uWSGI master process (pid: 80268) Aug 01 07:40:31.266699 dv-cinder-740146-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 07:40:31.271604 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: spawned uWSGI worker 1 (pid: 80280, cores: 1) Aug 01 07:40:31.271604 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: spawned uWSGI worker 2 (pid: 80281, cores: 1) Aug 01 07:40:31.271604 dv-cinder-740146-16-iscsi devstack@keystone.service[80268]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 07:40:36.870664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:40:36.870664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: from cryptography.utils import int_from_bytes Aug 01 07:40:36.871445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:40:36.871445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: from cryptography.utils import int_from_bytes Aug 01 07:40:36.882393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:40:36.882393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: from cryptography.utils import int_from_bytes Aug 01 07:40:36.882878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:40:36.882878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: from cryptography.utils import int_from_bytes Aug 01 07:40:36.923621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 07:40:36.923825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 07:40:36.924005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] command line args: [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 07:40:36.924320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 07:40:36.924667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ================================================================================ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 07:40:36.925109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] admin_token = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.925545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config_dir = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.925923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.926182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config_source = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.926370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] debug = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.926576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.927141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.927327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.927616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.928061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.928295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.928629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_config_append = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.928862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.929039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_dir = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.929164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.929604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.929870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.930126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.930509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.930681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.930999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.931384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.931823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.932220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.932469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.932711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.932952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.933173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 07:40:36.933273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.933345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 07:40:36.933444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.933551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] command line args: [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 07:40:36.933816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 07:40:36.934025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.934131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ================================================================================ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 07:40:36.934343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] admin_token = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.934520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.934634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config_dir = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.934773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.934936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] publish_errors = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.935038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config_source = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.935116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] debug = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.935317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.935463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.935726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.935821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.935963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.936218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.936426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.936572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.936711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.936835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.936909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_config_append = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.936969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.937027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.937146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] standard_threads = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.937252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_dir = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.937432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.937650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.937844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.937970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_journal = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_json = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.938915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_stderr = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.939039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.939142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_syslog = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.939214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.939274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.939332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.939442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.939505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.939614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.940213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.940323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.940480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.940715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.940979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.941075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.941220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.941335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.941485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.941713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.941804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.941960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.942216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.942309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.942421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] publish_errors = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.942526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.942719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.942884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.942994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.943196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.943339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.943439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.943708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.943961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.944121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.944218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.944446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.944702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.944848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.944946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.945177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.945438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.945531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.945703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.945959 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.946148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] standard_threads = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.946239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.946432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.946667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.946901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.947142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.947228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.947392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.947668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.947911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.948146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.948330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.948595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.948807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.948899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.949079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.949332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.949553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_journal = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.949637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.949793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.950035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.950271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.950411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_json = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.950513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.950729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.950885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_stderr = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.950977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.951052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_syslog = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.951175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.951288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:40:36.951477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.951699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.951784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.951934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.952038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.952261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.952506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.952589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.952745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.952827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.952989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.953084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.953228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.953307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.953478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.953622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.953703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.953878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.954922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.955073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.955155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.955329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.956049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.956177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.956287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.956395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.956584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.957195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.957366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.957541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.957666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.957789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.957944 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.958077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.958208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.958332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.958451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.958565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.958725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.958873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.959015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.959151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.959282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.959413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.959571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.959721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.959862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.960960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.961097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.961249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.961515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.961674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.961818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.961951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.962109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.962244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.962375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.962506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.962646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.962779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.962910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.963040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.963177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.963310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.963459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.963617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.963798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.964024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.964235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.964458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.964698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.964919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.965152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.965327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.965462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.965592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.965732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.965864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.965995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.966126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.966256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.966397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.966585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.966793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.967012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.967245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.967472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.968033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.968213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.968348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.968348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.968597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.968727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.968858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.969064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.969210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.969423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.969644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.969644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.970068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.970278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.970490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.970688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.970883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.external = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.971604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.971808 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.972132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.972347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.972566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.972776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.972978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.973185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.password = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.974044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.974308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.token = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.974308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.974901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.975170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.975489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.975799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.976067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.976267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.976618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.976618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.977160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.977160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.977627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.977769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.977902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.978036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.978354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.978542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.978724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.978862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.978995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.978995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.979454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.979685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.979860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.979997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.980137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.980434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.980623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.980802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.980992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.981145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.981287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.981557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.981747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.981931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.982130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.982325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.982506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.982687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.982687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.983056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.983243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.983388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.983548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.983689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.983821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.983965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.984334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.984483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.984624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.984757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.984889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.985315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.985455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.985588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.985734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.985868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.986000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.986131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.986263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.986395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.986536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.987427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.987636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.987774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.987906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.988311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.988732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.988925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.989095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.989289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.989479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.989666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.989666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.990028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.990200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.990406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.990606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.990797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.990986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.991169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.991350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.991599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.external = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.991758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.991957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.992155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.992343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.992531 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.992719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.992913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.993192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.password = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.993412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.993596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.token = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.993779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.993971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.994175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.994356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.994542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.994725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.994909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.995092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.995284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.995479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.995728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.995916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.996115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.996318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.996318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.996671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.996860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.997043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.997225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.997407 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.997589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.997790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.997790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.998146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.998336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.998528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.998528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.998892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.999094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.999266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.999457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.999664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:36.999664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.000023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.000213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.000371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.000506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.000649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.000782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.000922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.001968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.002100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.002274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.002452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.002655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.002864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.003073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.003296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.003517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.003768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.003962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.004100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.004235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.004368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.004500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.004632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.004773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.004917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.005948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.006079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.006219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.006353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.006485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.006627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.006781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.007070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.007235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.007235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.007507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.007507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.007797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.007951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.008124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.008273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.008442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.008654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.008901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.009058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.009201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.009335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.009477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.009609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.009742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.009913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.010077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.010214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.010384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.010525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.010696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.010907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.011106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.011287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.011287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.011605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.011605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.011947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.012096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.012273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.012489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.012701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.012868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.013006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.013166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.013314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.013470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.013740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.013906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.014048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.014211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.014352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.014501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.014642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.014777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.014910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.015068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.015255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.015394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.015567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.015721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.016191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.016444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.016646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.016844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.017031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.017172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.017328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.017328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.017620 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.017620 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.018555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.018770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.018988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.019234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.019455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.019712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.019936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.020159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.020387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.020634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.020773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.020952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.021201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.021364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.021514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.021702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.022093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.022319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.022546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.022769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.022990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.023330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.023818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.023998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.024194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.024382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.024784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.024983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.025173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.025338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.025487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.025619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.025761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.025894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.026244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.026381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.026513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.026646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.026793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.026934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.027135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.027439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.027603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.027756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.027974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.028111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.028244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.028377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.028509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.028641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.028789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.028924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.029057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.029198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.029330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.029464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.029603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.029753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.029886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.030018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.030018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.030267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.030408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.030542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.030721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.030861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.031009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.031149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.031308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.031442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.031627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.031815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.032005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.032191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.032427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.032597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.032810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.032810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.033185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.033373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.034168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.034496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.034666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.034666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.035005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.035005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.035300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.035445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.035612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.035768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.035768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.036031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.036183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.036389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.036569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.036753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.036753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.037162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.037357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.037545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.037688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.037828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.037967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.038114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.038254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.038400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.038602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.038774 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.038921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.039092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.039234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.039384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.039690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.039844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.040033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.040244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.040444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.040643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.040832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.041091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.041237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.041438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.041787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.041969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.042128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.042303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.042458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.042621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.042860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.043052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.043240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.043425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.043627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.043846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.044042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.044246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.044422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.044614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.044827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.045964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.046503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.046503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.046855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.046855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.047212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.047406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.047681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.047851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.048048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.048048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.048427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.048427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.048814 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.049026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.049255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.049457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.049655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.049869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.049869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.050386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.050694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.051752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.053329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.057348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.058916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.060257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.060257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.060257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.060257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.060257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.060257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:40:37.060257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 07:40:37.060613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.060613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.060613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.060788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.060788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.060788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.060958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.067026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.092110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.092438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.097366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.097600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.148716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.149230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.202420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.202420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.203172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.203411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.270262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.270431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.270628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.270849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.271067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.271283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.271651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.271892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.272121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.272346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.272569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.272787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.295652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.295865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.296969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.297056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.297262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.297479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.297698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.308489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.308982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.321903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.322129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.322352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.322575 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.322789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.322879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.323000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.323086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.323214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.323301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.323440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.323686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.323903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.324119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:40:37.340998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.341205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:40:37.341640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.341838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:40:37.341995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.342105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.342219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:40:37.342354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.342564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.342648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.343025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:40:37.343185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.343305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.343591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.344119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.344119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.353967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.354239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.354510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.354677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.354825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.354967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.355227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.355508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.361933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.362588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.366940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.367223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.367504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.367811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.368047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.368134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.368336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.368423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.368598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.368686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.368868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.368954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.369133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.369218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.369404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.369487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.369668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.369759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.369944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.370217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.370495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.370764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.394164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.395691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.395792 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.395973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.397275 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.397561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.400615 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.400897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.401176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.402225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.402506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.402789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.407990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.408273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.410244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.410547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.412473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.412766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.413046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.413326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.413599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.413881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.414157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.414439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.414713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.415005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.415301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.415609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.415886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.416166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.416435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.416723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.431025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.431382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.433848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.434133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.435325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.438110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.438515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.438818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.441987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.442280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.443865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.446918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.447031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.450209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.450356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.450652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.450938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.451292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.451604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.451896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.454331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.454635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.454926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.455225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.456152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.456453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.469380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.469663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.469929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.470189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.472921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.473724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.474013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.474293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.477583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.479117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.479390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.481346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.482360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.482634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.482906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.483031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.483183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.483324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.483641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.486060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.486340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.486667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.486961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.487254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.495322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.495623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.495983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.496263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.496550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.496821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.497093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.499442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.499745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.500028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.500311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.500596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.500876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.501165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.513499 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.516856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.518746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.520646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.523005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.523190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.524713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.525011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.525322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.526023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.531352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.531671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.531960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.532393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.534608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.537679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.537968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.538543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.538833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.540036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.541734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.544730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.545021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.545308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.545595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.546944 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.547231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.547509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.554278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.554574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.554859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.555241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.555518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.555818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.556094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.556376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.556648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.556928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.557195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.557564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.557846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.558137 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.558408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.558681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.558944 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.561796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:40:37.562079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.562362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.562652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.562928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.563211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.563495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.563796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.564167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.564456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.564775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.565043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.565330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.565596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:40:37.592652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a8d331f9c0 pid: 80280 (default app) Aug 01 07:40:37.596852 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f608-3333-4ef2-b7fb-017b26bef85f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:37.597140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f608-3333-4ef2-b7fb-017b26bef85f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:37.597415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f608-3333-4ef2-b7fb-017b26bef85f None None] PATH_INFO: `/v3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:37.600691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a8d331f9c0 pid: 80281 (default app) Aug 01 07:40:37.601750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Mon Aug 1 07:40:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:45.999448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e3f9-7774-40d0-ab8b-8c012bc1ee8a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:45.999988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e3f9-7774-40d0-ab8b-8c012bc1ee8a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:46.000069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e3f9-7774-40d0-ab8b-8c012bc1ee8a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:46.005051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:40:46.010286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:46.010681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:46.010792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:46.081832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: INFO dbcounter [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] Registered counter for database keystone Aug 01 07:40:46.101263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG oslo_db.sqlalchemy.engines [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80280) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 07:40:46.107428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writer thread running {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 07:40:46.293160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG passlib.handlers.bcrypt [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80280) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 07:40:46.293516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG passlib.handlers.bcrypt [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80280) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 07:40:46.349332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:46.351017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:46.484132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 01 07:40:46.607294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e3f9-7774-40d0-ab8b-8c012bc1ee8a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:46.676259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: INFO dbcounter [None req-e374e3f9-7774-40d0-ab8b-8c012bc1ee8a None None] Registered counter for database keystone Aug 01 07:40:46.697859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG oslo_db.sqlalchemy.engines [None req-e374e3f9-7774-40d0-ab8b-8c012bc1ee8a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80281) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 07:40:46.704036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writer thread running {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 07:40:46.825173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e3f9-7774-40d0-ab8b-8c012bc1ee8a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:46.829600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6e152-6091-4b8d-a286-af88ea061e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:46.830324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e152-6091-4b8d-a286-af88ea061e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:46.830645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e152-6091-4b8d-a286-af88ea061e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:46.830927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e152-6091-4b8d-a286-af88ea061e2f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:46.839000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6e152-6091-4b8d-a286-af88ea061e2f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:46.893415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING keystone.common.rbac_enforcer.enforcer [None req-99d6e152-6091-4b8d-a286-af88ea061e2f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 07:40:46.893763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6e152-6091-4b8d-a286-af88ea061e2f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:46.903420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 1 07:40:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:40:46.909431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:46.924397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd8b92-974f-47f5-832d-43dbd9c34bce None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:46.929069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552a1df-61ca-4c80-ad37-22fc8d089dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:46.930189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552a1df-61ca-4c80-ad37-22fc8d089dfa admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:46.930483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552a1df-61ca-4c80-ad37-22fc8d089dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:46.930762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552a1df-61ca-4c80-ad37-22fc8d089dfa admin admin] PATH_INFO: `/v3/projects/5b4d36fd27fd480aad590e2c1f973948` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:46.932305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552a1df-61ca-4c80-ad37-22fc8d089dfa admin admin] RBAC: Authorizing `identity:get_project(project_id=5b4d36fd27fd480aad590e2c1f973948)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:46.984523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f552a1df-61ca-4c80-ad37-22fc8d089dfa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 07:40:46.984890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552a1df-61ca-4c80-ad37-22fc8d089dfa admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:46.990491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Aug 1 07:40:46 2022] GET /identity/v3/projects/5b4d36fd27fd480aad590e2c1f973948 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:48.980635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b1daf-1c57-40c3-927e-c752e7a11b72 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:48.981194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b1daf-1c57-40c3-927e-c752e7a11b72 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:48.981194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b1daf-1c57-40c3-927e-c752e7a11b72 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:48.985318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:48.991236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b7f33-6c8e-4312-9e5a-9522fcbbb7ff None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:48.991694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b7f33-6c8e-4312-9e5a-9522fcbbb7ff None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:48.991838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b7f33-6c8e-4312-9e5a-9522fcbbb7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:49.012570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-456b7f33-6c8e-4312-9e5a-9522fcbbb7ff None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:49.014047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-456b7f33-6c8e-4312-9e5a-9522fcbbb7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:49.022069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 07:40:49.043445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b1daf-1c57-40c3-927e-c752e7a11b72 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:49.056162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b1daf-1c57-40c3-927e-c752e7a11b72 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:49.060404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe6674-d91a-49df-a261-a76f64d0ca41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:49.061012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6674-d91a-49df-a261-a76f64d0ca41 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:49.061300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6674-d91a-49df-a261-a76f64d0ca41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:49.061576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6674-d91a-49df-a261-a76f64d0ca41 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:49.069854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe6674-d91a-49df-a261-a76f64d0ca41 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:49.072079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe6674-d91a-49df-a261-a76f64d0ca41 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:49.080939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 1 07:40:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:40:49.086120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b7f33-6c8e-4312-9e5a-9522fcbbb7ff None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:49.095414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b7f33-6c8e-4312-9e5a-9522fcbbb7ff None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:49.099892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542c6e1-905c-46e0-9483-86f69225ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:49.100533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542c6e1-905c-46e0-9483-86f69225ae96 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:49.100897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542c6e1-905c-46e0-9483-86f69225ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:49.101188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542c6e1-905c-46e0-9483-86f69225ae96 admin admin] PATH_INFO: `/v3/users/3ab27936a00c428f91a5c67560723447` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:49.102570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542c6e1-905c-46e0-9483-86f69225ae96 admin admin] RBAC: Authorizing `identity:get_user(user_id=3ab27936a00c428f91a5c67560723447)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:49.104450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542c6e1-905c-46e0-9483-86f69225ae96 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:49.105990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Mon Aug 1 07:40:49 2022] GET /identity/v3/users/3ab27936a00c428f91a5c67560723447 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:51.178941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7365f7d-a305-49df-b7f3-c4276c79a2b7 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.179491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7365f7d-a305-49df-b7f3-c4276c79a2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.179491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7365f7d-a305-49df-b7f3-c4276c79a2b7 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.181063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:40:51.181666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc32b22-7ed4-4e32-ad88-713580628972 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.182132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc32b22-7ed4-4e32-ad88-713580628972 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.182227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc32b22-7ed4-4e32-ad88-713580628972 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.184081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:40:51.186055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.186354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.186634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.189723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0975636-7149-4001-bb72-c204698601d7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.190030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0975636-7149-4001-bb72-c204698601d7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.190316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0975636-7149-4001-bb72-c204698601d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.210050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c0975636-7149-4001-bb72-c204698601d7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:51.211268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c0975636-7149-4001-bb72-c204698601d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:51.230142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG passlib.handlers.bcrypt [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80281) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 07:40:51.230571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG passlib.handlers.bcrypt [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80281) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 07:40:51.276595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:51.278186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:51.290821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 7/12] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:40:51.312670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0975636-7149-4001-bb72-c204698601d7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.336416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.381398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 6/13] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:40:51.403816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.419470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0975636-7149-4001-bb72-c204698601d7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.453230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.453798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.454078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.454350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.506499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23412ef1-fef9-471e-89d3-d0095a753624 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.508846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.522081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.523695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.546237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.547359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.547685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.547891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.547986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.560361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:40:51.569227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:40:51.570262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:40:51.571107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:40:51.576312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.624151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 8/15] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:40:51.630336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.677006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaed12-3685-4fa8-81a5-6bce5cb0c356 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.711706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.712262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.712552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.712833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.713566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:51.713877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.715317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.727382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 8/16] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:51.729106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea695ff-020c-4bf9-babe-b2ae34282704 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.732870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.751150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.763648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.764482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.768710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.768710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.768710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.777426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.778921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:40:51.797169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:40:51.798176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:40:51.799279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:40:51.799279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:40:51.799279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:40:51.799279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:40:51.799279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:40:51.804120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.838134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1edd4b-1056-4080-875e-54a54b7374f4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.870625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.871273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.871580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.871857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] PATH_INFO: `/v3/users/3ab27936a00c428f91a5c67560723447` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.873354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] RBAC: Authorizing `identity:get_user(user_id=3ab27936a00c428f91a5c67560723447)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.874817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.876926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/users/3ab27936a00c428f91a5c67560723447 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:51.882369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.905929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3182-cf93-45c2-a2a2-358d996e17ef None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.940771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df95df4-ca77-4adf-b4be-80abdcc47ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.941399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df95df4-ca77-4adf-b4be-80abdcc47ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.941686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df95df4-ca77-4adf-b4be-80abdcc47ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.941968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df95df4-ca77-4adf-b4be-80abdcc47ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.942672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df95df4-ca77-4adf-b4be-80abdcc47ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:51.942985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df95df4-ca77-4adf-b4be-80abdcc47ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.944428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df95df4-ca77-4adf-b4be-80abdcc47ae9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.977950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.984847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c050a22-f00a-4d42-a0df-4cf1ae9fdf8f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:52.022872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:52.023573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:52.023861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:52.024139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:52.025356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:52.026865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:52.028563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:52.035255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:52.068919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:52.141220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324077d-41b9-4398-81bc-ccf8ada8aa87 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:52.178898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7f63a-4d7c-45a8-a306-6f12893dda5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:52.179569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f63a-4d7c-45a8-a306-6f12893dda5d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:52.180116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f63a-4d7c-45a8-a306-6f12893dda5d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:52.180116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f63a-4d7c-45a8-a306-6f12893dda5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:52.180925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f63a-4d7c-45a8-a306-6f12893dda5d None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.domain.id=default, user.id=3ab27936a00c428f91a5c67560723447) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:52.181251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f63a-4d7c-45a8-a306-6f12893dda5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:52.182735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f63a-4d7c-45a8-a306-6f12893dda5d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:52.192496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 1 07:40:52 2022] GET /identity/v3/role_assignments?user.id=3ab27936a00c428f91a5c67560723447&scope.domain.id=default&role.id=05421dac98a848f68d13ca21036dbfbb => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:54.518377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe527-2f64-4d73-8c22-cf2182d4950f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:54.518377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe527-2f64-4d73-8c22-cf2182d4950f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:54.518377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe527-2f64-4d73-8c22-cf2182d4950f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:54.518377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:54.524784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ef882-3c84-4264-93b7-2e492ed39f71 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:54.525510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ef882-3c84-4264-93b7-2e492ed39f71 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:54.525933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ef882-3c84-4264-93b7-2e492ed39f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:54.557492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-766ef882-3c84-4264-93b7-2e492ed39f71 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:54.558912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-766ef882-3c84-4264-93b7-2e492ed39f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:54.592676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:54.647948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 12/23] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:40:54.674646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efe527-2f64-4d73-8c22-cf2182d4950f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:54.731018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:54.817779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efe527-2f64-4d73-8c22-cf2182d4950f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:54.857275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366acb5-1b88-4957-8146-3092237544df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:54.857933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366acb5-1b88-4957-8146-3092237544df None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:54.858211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366acb5-1b88-4957-8146-3092237544df None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:54.858486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366acb5-1b88-4957-8146-3092237544df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:54.875313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366acb5-1b88-4957-8146-3092237544df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:54.876939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366acb5-1b88-4957-8146-3092237544df None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1366acb5-1b88-4957-8146-3092237544df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:40:54.896378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:40:54.897239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:40:54.898044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:40:54.898044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:40:54.898044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:40:54.898044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:40:54.898044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 12/24] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:40:54.902975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7fbca-8713-449a-9b31-a921d929f6da None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:54.903269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7fbca-8713-449a-9b31-a921d929f6da None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:54.903556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7fbca-8713-449a-9b31-a921d929f6da None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:54.905055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 13/25] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:54.905782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366acb5-1b88-4957-8146-3092237544df None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:54.909295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e14de8-fede-4336-80ca-208dd28690db None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:54.909812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e14de8-fede-4336-80ca-208dd28690db None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:54.910316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e14de8-fede-4336-80ca-208dd28690db None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:54.912506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 14/26] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:54.913761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536a173-93a4-477a-8e21-53d444646903 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:54.914268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536a173-93a4-477a-8e21-53d444646903 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:54.914736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536a173-93a4-477a-8e21-53d444646903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:54.952986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b536a173-93a4-477a-8e21-53d444646903 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:54.954711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b536a173-93a4-477a-8e21-53d444646903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:55.014778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:55.051253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366acb5-1b88-4957-8146-3092237544df None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:55.058398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 15/27] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:40:55.059720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15209598-118a-4d90-8de8-1ea518203504 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.060176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15209598-118a-4d90-8de8-1ea518203504 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.060599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15209598-118a-4d90-8de8-1ea518203504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.080307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-15209598-118a-4d90-8de8-1ea518203504 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:55.081748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-15209598-118a-4d90-8de8-1ea518203504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:55.124285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:55.125014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.125500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.125777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.126687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:55.127234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:55.128846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:55.141342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 13/28] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:55.143296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:55.159678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 16/29] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:40:55.161266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffdb72-82e0-4d36-b6bc-48c4f551689a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.161779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffdb72-82e0-4d36-b6bc-48c4f551689a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.162240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffdb72-82e0-4d36-b6bc-48c4f551689a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.164234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 17/30] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:55.165118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffdb72-82e0-4d36-b6bc-48c4f551689a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:55.195957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:55.266297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44476ff-16a4-418c-894d-22e6945e1df2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:55.307965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158a440-cbb4-42f4-b413-c5c164d48e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:55.308970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158a440-cbb4-42f4-b413-c5c164d48e7d None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.309457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158a440-cbb4-42f4-b413-c5c164d48e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.309964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158a440-cbb4-42f4-b413-c5c164d48e7d None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.310980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158a440-cbb4-42f4-b413-c5c164d48e7d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:55.312776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158a440-cbb4-42f4-b413-c5c164d48e7d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:55.398604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffdb72-82e0-4d36-b6bc-48c4f551689a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:55.443346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae46639-f0ac-4eec-ae87-a6c8cb276757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:55.444022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae46639-f0ac-4eec-ae87-a6c8cb276757 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.444311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae46639-f0ac-4eec-ae87-a6c8cb276757 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.444644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae46639-f0ac-4eec-ae87-a6c8cb276757 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.445399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae46639-f0ac-4eec-ae87-a6c8cb276757 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:55.446859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae46639-f0ac-4eec-ae87-a6c8cb276757 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:55.543722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 14/31] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/roles => generated 219 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:55.545164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9372-d642-4a16-a085-228abc9f6ef7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.545460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9372-d642-4a16-a085-228abc9f6ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.545744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9372-d642-4a16-a085-228abc9f6ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.561074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:55.567849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-98cf9372-d642-4a16-a085-228abc9f6ef7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:55.569644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-98cf9372-d642-4a16-a085-228abc9f6ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:55.661772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 15/32] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:40:55.663083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf9372-d642-4a16-a085-228abc9f6ef7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:55.716980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 18/33] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/projects => generated 301 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:55.719620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8994ce6-2e97-44bb-8e90-3614d9701bef None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.720736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8994ce6-2e97-44bb-8e90-3614d9701bef None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.721653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8994ce6-2e97-44bb-8e90-3614d9701bef None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.723862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 19/34] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:55.725606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8994ce6-2e97-44bb-8e90-3614d9701bef None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:55.833739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf9372-d642-4a16-a085-228abc9f6ef7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:55.843382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:55.883491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8994ce6-2e97-44bb-8e90-3614d9701bef None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:55.887438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:55.888132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.888415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.888699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.889504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:55.891070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:55.897277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:55.925823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fa32f-a529-4c90-ae26-264499c5fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:55.926397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fa32f-a529-4c90-ae26-264499c5fd39 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.926692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fa32f-a529-4c90-ae26-264499c5fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.926975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fa32f-a529-4c90-ae26-264499c5fd39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.938837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fa32f-a529-4c90-ae26-264499c5fd39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-024fa32f-a529-4c90-ae26-264499c5fd39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:40:55.944892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:40:55.949935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 20/35] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:40:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:40:55.950768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bc084-4fe9-4d01-a282-e6ef54d52996 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.951573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bc084-4fe9-4d01-a282-e6ef54d52996 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.951573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bc084-4fe9-4d01-a282-e6ef54d52996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.969012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-406bc084-4fe9-4d01-a282-e6ef54d52996 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:55.971277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-406bc084-4fe9-4d01-a282-e6ef54d52996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:55.977399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 16/36] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/roles => generated 225 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:55.978878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.053697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 21/37] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:40:56.076157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bc084-4fe9-4d01-a282-e6ef54d52996 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.134454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d037700-6b14-4299-9f12-41c82df3ae79 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.144663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:56.183594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a690e-d384-41f1-94e3-9445fd81484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.184512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a690e-d384-41f1-94e3-9445fd81484f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.184791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a690e-d384-41f1-94e3-9445fd81484f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.185072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a690e-d384-41f1-94e3-9445fd81484f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.185810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a690e-d384-41f1-94e3-9445fd81484f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:56.186125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a690e-d384-41f1-94e3-9445fd81484f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.187603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a690e-d384-41f1-94e3-9445fd81484f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.198664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bc084-4fe9-4d01-a282-e6ef54d52996 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.205185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 17/38] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:40:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:56.206614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af7fb-5cb8-4ce8-980b-5657dceffaaa None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.206916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af7fb-5cb8-4ce8-980b-5657dceffaaa None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.207199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af7fb-5cb8-4ce8-980b-5657dceffaaa None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.208642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 18/39] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:56.213594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af7fb-5cb8-4ce8-980b-5657dceffaaa None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.243204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561221d-802c-41f6-b207-e900e589af6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.243916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561221d-802c-41f6-b207-e900e589af6b None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.244205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561221d-802c-41f6-b207-e900e589af6b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.244577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561221d-802c-41f6-b207-e900e589af6b None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.245306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561221d-802c-41f6-b207-e900e589af6b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.246817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561221d-802c-41f6-b207-e900e589af6b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.277391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:56.311930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 22/40] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 07:40:56 2022] POST /identity/v3/roles => generated 223 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:40:56.311930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986707-da0c-4864-9808-c0bbaa2b5d51 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.311930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986707-da0c-4864-9808-c0bbaa2b5d51 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.311930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986707-da0c-4864-9808-c0bbaa2b5d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.323955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af7fb-5cb8-4ce8-980b-5657dceffaaa None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.338693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-65986707-da0c-4864-9808-c0bbaa2b5d51 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:56.339982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-65986707-da0c-4864-9808-c0bbaa2b5d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:56.362027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.362793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.363158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.363447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] PATH_INFO: `/v3/users/3ab27936a00c428f91a5c67560723447` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.364802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] RBAC: Authorizing `identity:get_user(user_id=3ab27936a00c428f91a5c67560723447)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.366373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.368383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 19/41] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:40:56 2022] GET /identity/v3/users/3ab27936a00c428f91a5c67560723447 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:56.373850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.418875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 23/42] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:40:56.443004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65986707-da0c-4864-9808-c0bbaa2b5d51 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.494732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75cd10-9385-4243-9d6c-fd8342fef8de None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.499740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:56.525390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:56.534257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.534918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.535208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.535491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.536686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.538159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.540083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 20/43] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:40:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:56.545684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.558255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65986707-da0c-4864-9808-c0bbaa2b5d51 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.592444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.593097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.593384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.593673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.594831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.596461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.598057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 24/44] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:40:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:56.603623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.647950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e79ac-8e07-4f1a-a1d2-6b29e50b0c07 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.681868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48812bd1-0eda-4446-bd8b-96e9bd5b7053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.682651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48812bd1-0eda-4446-bd8b-96e9bd5b7053 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.682942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48812bd1-0eda-4446-bd8b-96e9bd5b7053 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.683235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48812bd1-0eda-4446-bd8b-96e9bd5b7053 None admin] PATH_INFO: `/v3/domains/default/users/3ab27936a00c428f91a5c67560723447/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.685296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48812bd1-0eda-4446-bd8b-96e9bd5b7053 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3ab27936a00c428f91a5c67560723447, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.686776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48812bd1-0eda-4446-bd8b-96e9bd5b7053 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.705657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ada0d-0eec-4fff-981e-6f887af85a94 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.721580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:56.739557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce912b-1766-4fe0-a28a-9f78b10279dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.740223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce912b-1766-4fe0-a28a-9f78b10279dc None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.740518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce912b-1766-4fe0-a28a-9f78b10279dc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.740798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce912b-1766-4fe0-a28a-9f78b10279dc None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.741487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce912b-1766-4fe0-a28a-9f78b10279dc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.742935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce912b-1766-4fe0-a28a-9f78b10279dc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.763566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 21/45] 10.222.0.48 () {58 vars in 1361 bytes} [Mon Aug 1 07:40:56 2022] PUT /identity/v3/domains/default/users/3ab27936a00c428f91a5c67560723447/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:40:56.822800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 25/46] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:40:56 2022] POST /identity/v3/projects => generated 312 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:40:58.777611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93b12e-7ec3-406c-b1cb-cde68e33116f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:58.778087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93b12e-7ec3-406c-b1cb-cde68e33116f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:58.778170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93b12e-7ec3-406c-b1cb-cde68e33116f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:58.779551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 22/47] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:58.784855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dbdc0-0789-4d15-9b2d-891925806d6e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:58.785266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dbdc0-0789-4d15-9b2d-891925806d6e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:58.785416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dbdc0-0789-4d15-9b2d-891925806d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:58.803482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-623dbdc0-0789-4d15-9b2d-891925806d6e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:58.804719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-623dbdc0-0789-4d15-9b2d-891925806d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:58.818526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e47e2-214f-44bf-8f72-d71441aa93a1 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:58.818815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e47e2-214f-44bf-8f72-d71441aa93a1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:58.819090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e47e2-214f-44bf-8f72-d71441aa93a1 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:58.820548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 23/48] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:40:58.825190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b032f5a-c9ed-475b-aea3-f86bec59b3f8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:58.825484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b032f5a-c9ed-475b-aea3-f86bec59b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:58.825764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b032f5a-c9ed-475b-aea3-f86bec59b3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:58.843728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5b032f5a-c9ed-475b-aea3-f86bec59b3f8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:58.844946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5b032f5a-c9ed-475b-aea3-f86bec59b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:58.873193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 26/49] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:40:58.895497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:58.896052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dbdc0-0789-4d15-9b2d-891925806d6e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:58.912881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 24/50] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:40:58.936072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b032f5a-c9ed-475b-aea3-f86bec59b3f8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:58.992285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:58.996078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dbdc0-0789-4d15-9b2d-891925806d6e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:59.030300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:59.030882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:59.031170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:59.031454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:59.036742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b032f5a-c9ed-475b-aea3-f86bec59b3f8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:59.041634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:40:59.046935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:40:59.047809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:40:59.049088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 27/51] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:40:59.055711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:59.071915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:59.072569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:59.072860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:59.073145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:59.074320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:59.075835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:59.077435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 25/52] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:40:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:59.083126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:59.136507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:59.157713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11532399-72ca-4062-a94e-072b4d49c233 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:59.183337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75751-29f0-4160-a542-b242ff604d24 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:59.191369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:59.192024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:59.192310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:59.192586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:59.193306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:59.193615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:59.195013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:59.203551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 28/53] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:40:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:59.209797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:59.219027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:59.219610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:59.219939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:59.220217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:59.220914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:59.222390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:59.231731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:59.308307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbb6d0-f0e8-4cfd-b69f-23983eb0b5e7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:59.336764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 26/54] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 1 07:40:59 2022] POST /identity/v3/projects => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:40:59.344468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:59.345682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:59.346007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:59.346292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] PATH_INFO: `/v3/users/3ab27936a00c428f91a5c67560723447` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:59.347633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] RBAC: Authorizing `identity:get_user(user_id=3ab27936a00c428f91a5c67560723447)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:59.349161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:59.351165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 29/55] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:40:59 2022] GET /identity/v3/users/3ab27936a00c428f91a5c67560723447 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:59.356998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:59.469451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9063034-1a3b-4e47-a6f0-186b48b3b954 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:59.472001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:59.505938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da105-1502-46ff-932c-13ad7bd7e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:59.506598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da105-1502-46ff-932c-13ad7bd7e844 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:59.506888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da105-1502-46ff-932c-13ad7bd7e844 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:59.507169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da105-1502-46ff-932c-13ad7bd7e844 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:59.508400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299da105-1502-46ff-932c-13ad7bd7e844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:59.509906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299da105-1502-46ff-932c-13ad7bd7e844 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:59.511732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 27/56] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:40:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:59.517669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:59.605300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:59.617382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7749a9c-297d-4373-8630-6737c110aa97 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:59.651860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062a0f5-dcdd-4bd7-a293-71f808c493b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:59.652483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062a0f5-dcdd-4bd7-a293-71f808c493b1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:59.652766 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062a0f5-dcdd-4bd7-a293-71f808c493b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:59.653042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062a0f5-dcdd-4bd7-a293-71f808c493b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:59.653882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062a0f5-dcdd-4bd7-a293-71f808c493b1 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.domain.id=default, user.id=3ab27936a00c428f91a5c67560723447) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:59.654139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062a0f5-dcdd-4bd7-a293-71f808c493b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:59.655598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062a0f5-dcdd-4bd7-a293-71f808c493b1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:59.661474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 30/57] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 1 07:40:59 2022] GET /identity/v3/role_assignments?user.id=3ab27936a00c428f91a5c67560723447&scope.domain.id=default&role.id=05421dac98a848f68d13ca21036dbfbb => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:01.350052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56adcad-9421-4602-9036-3c815b8e9e0e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:01.350563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56adcad-9421-4602-9036-3c815b8e9e0e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:01.350646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56adcad-9421-4602-9036-3c815b8e9e0e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:01.352287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 28/58] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:01.357263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c41bfc-bc65-42aa-bd69-5d9a84016552 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:01.357656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c41bfc-bc65-42aa-bd69-5d9a84016552 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:01.357831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c41bfc-bc65-42aa-bd69-5d9a84016552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:01.376291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c6c41bfc-bc65-42aa-bd69-5d9a84016552 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:01.377491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c6c41bfc-bc65-42aa-bd69-5d9a84016552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:01.442367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 31/59] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:01.465540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56adcad-9421-4602-9036-3c815b8e9e0e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:01.566389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56adcad-9421-4602-9036-3c815b8e9e0e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:01.599338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f728e3b-795d-446f-bb75-351abc259683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:01.599995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f728e3b-795d-446f-bb75-351abc259683 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:01.600337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f728e3b-795d-446f-bb75-351abc259683 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:01.600571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f728e3b-795d-446f-bb75-351abc259683 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:01.601730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f728e3b-795d-446f-bb75-351abc259683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:01.603197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f728e3b-795d-446f-bb75-351abc259683 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:01.604793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 29/60] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:01.610891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c41bfc-bc65-42aa-bd69-5d9a84016552 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:01.692892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:01.710701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c41bfc-bc65-42aa-bd69-5d9a84016552 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:01.744413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b204ca-b010-4b94-9097-c1253faf4693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:01.745073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b204ca-b010-4b94-9097-c1253faf4693 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:01.745349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b204ca-b010-4b94-9097-c1253faf4693 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:01.745629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b204ca-b010-4b94-9097-c1253faf4693 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:01.746316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b204ca-b010-4b94-9097-c1253faf4693 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:01.747805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b204ca-b010-4b94-9097-c1253faf4693 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:01.842653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 32/61] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:01 2022] POST /identity/v3/users => generated 275 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:04.842491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21d197-93d5-4caf-9662-79fe3a9c873a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:04.843477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21d197-93d5-4caf-9662-79fe3a9c873a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:04.844010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21d197-93d5-4caf-9662-79fe3a9c873a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:04.845890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 30/62] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:04.850777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12595499-e32b-4af4-bb9a-b262a5647ac9 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:04.851609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12595499-e32b-4af4-bb9a-b262a5647ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:04.852093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12595499-e32b-4af4-bb9a-b262a5647ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:04.880538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-12595499-e32b-4af4-bb9a-b262a5647ac9 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:04.882104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-12595499-e32b-4af4-bb9a-b262a5647ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:04.972866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 33/63] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:05.008512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21d197-93d5-4caf-9662-79fe3a9c873a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.064177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:05.139097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21d197-93d5-4caf-9662-79fe3a9c873a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.154550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608eaba-2b46-42fd-9fd3-fe9c9662bf78 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.155495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608eaba-2b46-42fd-9fd3-fe9c9662bf78 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.155918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608eaba-2b46-42fd-9fd3-fe9c9662bf78 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.157972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 34/64] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:41:05.163438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e774908-eb15-444a-b246-6a23ca64aac8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.164058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e774908-eb15-444a-b246-6a23ca64aac8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.164435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e774908-eb15-444a-b246-6a23ca64aac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.177643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.178297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.178585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.178865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.187421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4e774908-eb15-444a-b246-6a23ca64aac8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:05.189516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e774908-eb15-444a-b246-6a23ca64aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:05.200174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:05.212635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:05.213560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:05.214334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 31/65] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:05.224564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.266653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:05.273564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 35/66] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:05.314133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e774908-eb15-444a-b246-6a23ca64aac8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.413571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:05.417008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ec557-4596-4cb4-a9d2-1c5fbef7393c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.437587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e774908-eb15-444a-b246-6a23ca64aac8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.459868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270de3d-9491-4057-8c45-bad0d91fadf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.460911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270de3d-9491-4057-8c45-bad0d91fadf1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.461448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270de3d-9491-4057-8c45-bad0d91fadf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.461956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270de3d-9491-4057-8c45-bad0d91fadf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.462861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270de3d-9491-4057-8c45-bad0d91fadf1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:05.463722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270de3d-9491-4057-8c45-bad0d91fadf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:05.465523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270de3d-9491-4057-8c45-bad0d91fadf1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:05.477957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 32/67] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:05.479233 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39e5ef-91c5-4e80-be67-a649ffd0f58a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.479554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39e5ef-91c5-4e80-be67-a649ffd0f58a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.479654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c19fd-9edf-47b6-88c7-c3974b92b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.479834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39e5ef-91c5-4e80-be67-a649ffd0f58a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.480466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c19fd-9edf-47b6-88c7-c3974b92b852 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.480826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c19fd-9edf-47b6-88c7-c3974b92b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.481183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c19fd-9edf-47b6-88c7-c3974b92b852 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.481327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 33/68] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:05.482549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91859b-74c7-40a4-acc4-b79363dc16ec None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.482841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91859b-74c7-40a4-acc4-b79363dc16ec None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.483124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91859b-74c7-40a4-acc4-b79363dc16ec None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.484437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 34/69] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:05.486514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91859b-74c7-40a4-acc4-b79363dc16ec None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.495677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c19fd-9edf-47b6-88c7-c3974b92b852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-871c19fd-9edf-47b6-88c7-c3974b92b852 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:05.501038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:05.502103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:05.503143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:05.504391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 36/70] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:05.506888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7882d-91d8-42e0-a95b-96f12f0151cd None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.507189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7882d-91d8-42e0-a95b-96f12f0151cd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.507472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7882d-91d8-42e0-a95b-96f12f0151cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.525376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a2a7882d-91d8-42e0-a95b-96f12f0151cd None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:05.526579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a2a7882d-91d8-42e0-a95b-96f12f0151cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:05.585918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:05.600991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 37/71] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:05.602241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0a9ac-7ee8-4589-8825-2c480fa433df None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.602537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0a9ac-7ee8-4589-8825-2c480fa433df None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.602818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0a9ac-7ee8-4589-8825-2c480fa433df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.604942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91859b-74c7-40a4-acc4-b79363dc16ec None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.621447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-33d0a9ac-7ee8-4589-8825-2c480fa433df None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:05.622906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-33d0a9ac-7ee8-4589-8825-2c480fa433df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:05.649648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.650302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.650592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.651114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.665994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:05.667412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:05.669286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 35/72] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:05.674351 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.695623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 38/73] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:05.696252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:05.696361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0a9ac-7ee8-4589-8825-2c480fa433df None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.780640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f6fee-caee-4bb3-b3e6-38efd04fc2cd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.801280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0a9ac-7ee8-4589-8825-2c480fa433df None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.815385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.816055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.816342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.816621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.817337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:05.817647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:05.819058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:05.819178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:05.827016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 36/74] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:05.835612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.836643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11575ff3-e420-41fc-8987-f3742d026041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.837294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575ff3-e420-41fc-8987-f3742d026041 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.837577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575ff3-e420-41fc-8987-f3742d026041 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.837850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575ff3-e420-41fc-8987-f3742d026041 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.848893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11575ff3-e420-41fc-8987-f3742d026041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-11575ff3-e420-41fc-8987-f3742d026041 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:05.853812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:05.855091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:05.855886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 39/75] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:05.856838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f492-9197-4254-9c72-7655a1985a0f None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.857051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f492-9197-4254-9c72-7655a1985a0f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.857344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f492-9197-4254-9c72-7655a1985a0f None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.858580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 40/76] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:05.859320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f492-9197-4254-9c72-7655a1985a0f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:05.919674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:05.937497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a48e91-b97e-48a6-a33d-73b703c0e947 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.960681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f492-9197-4254-9c72-7655a1985a0f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:05.972385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.973029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.973321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.973606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:05.993552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:05.994194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:05.994477 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:05.994761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.005235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:06.010117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:06.011018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 41/77] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:06.012058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.041800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.043292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.053162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.114601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217c2f-4f9a-4620-b002-566aa9d37222 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.122624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.142165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 37/78] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:06.142859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.150019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19beaf12-bf71-4998-96a5-acb570efe4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.150676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beaf12-bf71-4998-96a5-acb570efe4fe None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.150965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beaf12-bf71-4998-96a5-acb570efe4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.151243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beaf12-bf71-4998-96a5-acb570efe4fe None admin] PATH_INFO: `/v3/users/3ab27936a00c428f91a5c67560723447` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.152471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19beaf12-bf71-4998-96a5-acb570efe4fe None admin] RBAC: Authorizing `identity:get_user(user_id=3ab27936a00c428f91a5c67560723447)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.153935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19beaf12-bf71-4998-96a5-acb570efe4fe None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.155550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 42/79] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/users/3ab27936a00c428f91a5c67560723447 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.156867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f8c5e-49f6-4ac6-8e51-5413458f658b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.157162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f8c5e-49f6-4ac6-8e51-5413458f658b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.157446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f8c5e-49f6-4ac6-8e51-5413458f658b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.175469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f67f8c5e-49f6-4ac6-8e51-5413458f658b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:06.176667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f67f8c5e-49f6-4ac6-8e51-5413458f658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:06.243786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 43/80] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:06.244620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f8c5e-49f6-4ac6-8e51-5413458f658b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.247798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ce3a-ab2b-4bb8-b150-13432916f3b6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.278240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.283716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.284353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.284645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.284927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.285650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:06.285957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.287358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.295416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 38/81] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.296249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.342931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f8c5e-49f6-4ac6-8e51-5413458f658b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.343712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.379034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.379698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.379986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.380257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.380980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:06.381563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.382674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.390712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 44/82] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.392413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.398952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2dfd-48e6-4bcc-9482-6d8aad92d2f8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.432614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.433194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.433483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.433764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.435271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.436799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.439188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 39/83] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.440145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.489338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.493124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cdc3f-197a-415b-820e-6b40e7080339 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.527589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.528214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.528502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.529171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.540933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa5a68-ec67-4e70-a5e3-a38592a98d0b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.553501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.573803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.574450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.574735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.575022 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.576279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.577759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.580434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 40/84] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.581467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.598217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.599711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.682122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87644002-3b19-4576-80b5-0de8aec8f6e2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.693789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.697863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 45/85] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:06.698662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.721352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.722031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.722322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.722605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.723834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.725305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.726940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 41/86] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.727728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.779675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.801526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee791fb-2ab1-46ba-8e31-6bb1f52441a4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.831248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c44183-6680-4534-ac46-9f5978b8cb1e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.835732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.836383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.836672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.836952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.838128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.839626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.841248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 46/87] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.842242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.864980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:06.865600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:06.865885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:06.866164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:06.866957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=5bbed68b31dc4a55a7039cbb96dc0c97) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:06.867269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:06.868760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:06.875313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 42/88] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/role_assignments?user.id=5bbed68b31dc4a55a7039cbb96dc0c97&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:06.876660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:06.921174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:06.959360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28fc0b-aa20-4a2f-84cc-7d1831cbcb49 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:06.995672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17da4e4-a04c-4427-a50c-21f0939767c7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:07.002759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:07.004431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:07.004952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:07.005419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:07.006822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:07.009284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:07.010747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:07.035700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:07.036349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:07.036642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:07.036922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:07.038124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:07.039702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:07.041554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 43/89] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:06 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:07.045266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:07.088144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 47/90] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:06 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:07.089444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:07.181548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3174-3fbf-47d8-8f84-d72bf3946f76 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:07.232417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:07.232955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2216255-dd56-4e4b-bc39-4933ed628e4d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:07.233149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:07.233542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:07.233890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:07.261565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:07.279955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:07.280531 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:07.280823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:07.281182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:07.301994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:07.325444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:07.327988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:07.375016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:07.378981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:07.467882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 44/91] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:07.469430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:07.510860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 48/92] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:07.512387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:07.555397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:07.598686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:07.630761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b00d4-3295-4078-9cbc-cf45bf921322 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:07.702448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389c028-bff3-4731-8d9c-520e9a46d3a4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:07.705052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:07.705680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:07.705962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:07.706242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:07.707038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=3ab27936a00c428f91a5c67560723447) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:07.707349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:07.710082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:07.725753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 45/93] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:07 2022] GET /identity/v3/role_assignments?user.id=3ab27936a00c428f91a5c67560723447&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:07.726597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:07.796278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:07.796960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:07.797251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:07.798075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:07.799337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:07.800839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:07.802542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 49/94] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:07 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:07.809665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:07.929818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111037f-2e46-48f6-867a-b193faf3b704 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:07.963024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:07.973921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef2aa0-a126-40be-9866-67e2c9445507 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:08.016264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:08.016978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:08.017255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:08.017561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] PATH_INFO: `/v3/projects/548ed9cc5ec440269d1066d88287d7fb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:08.020070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] RBAC: Authorizing `identity:get_project(project_id=548ed9cc5ec440269d1066d88287d7fb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:08.021493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:08.023370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 46/95] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:07 2022] GET /identity/v3/projects/548ed9cc5ec440269d1066d88287d7fb => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:08.029234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:08.052023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea1f28-a524-4577-bd96-99efa41ef8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:08.052023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea1f28-a524-4577-bd96-99efa41ef8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:08.052023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea1f28-a524-4577-bd96-99efa41ef8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:08.052023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea1f28-a524-4577-bd96-99efa41ef8b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:08.052705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea1f28-a524-4577-bd96-99efa41ef8b0 None admin] RBAC: Adding query filter params (role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=5bbed68b31dc4a55a7039cbb96dc0c97) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:08.053025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea1f28-a524-4577-bd96-99efa41ef8b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:08.056280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea1f28-a524-4577-bd96-99efa41ef8b0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:08.056409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:08.068053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 50/96] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:07 2022] GET /identity/v3/role_assignments?user.id=5bbed68b31dc4a55a7039cbb96dc0c97&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:08.166346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61b2e3-cf8c-44ec-8a51-5a20928e05e6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:08.218298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5d5a3-dee9-4389-a8fb-ab8cade7276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:08.218946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5d5a3-dee9-4389-a8fb-ab8cade7276e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:08.219784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5d5a3-dee9-4389-a8fb-ab8cade7276e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:08.220073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5d5a3-dee9-4389-a8fb-ab8cade7276e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:08.220904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5d5a3-dee9-4389-a8fb-ab8cade7276e None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=548ed9cc5ec440269d1066d88287d7fb, user.id=5bbed68b31dc4a55a7039cbb96dc0c97) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:08.221220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5d5a3-dee9-4389-a8fb-ab8cade7276e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:08.222713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5d5a3-dee9-4389-a8fb-ab8cade7276e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:08.232197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 47/97] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:08 2022] GET /identity/v3/role_assignments?user.id=5bbed68b31dc4a55a7039cbb96dc0c97&scope.project.id=548ed9cc5ec440269d1066d88287d7fb&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:09.664051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372eab5-8ea8-42de-b78d-a55ed7c45e7e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:09.664573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372eab5-8ea8-42de-b78d-a55ed7c45e7e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:09.664683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372eab5-8ea8-42de-b78d-a55ed7c45e7e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:09.666032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 51/98] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:09.671906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df0369-7317-4ecb-a4c2-afc36e8dfad1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:09.672423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df0369-7317-4ecb-a4c2-afc36e8dfad1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:09.672613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df0369-7317-4ecb-a4c2-afc36e8dfad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:09.698170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f2df0369-7317-4ecb-a4c2-afc36e8dfad1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:09.699395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f2df0369-7317-4ecb-a4c2-afc36e8dfad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:09.786097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 48/99] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:09.816423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372eab5-8ea8-42de-b78d-a55ed7c45e7e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:09.886287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e673a-6554-459d-bc02-0e8c332ae3aa None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:09.886644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e673a-6554-459d-bc02-0e8c332ae3aa None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:09.886857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e673a-6554-459d-bc02-0e8c332ae3aa None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:09.888682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 49/100] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:41:09.895092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb380f-ae57-4860-b29f-7adb96e71519 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:09.895394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb380f-ae57-4860-b29f-7adb96e71519 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:09.895712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb380f-ae57-4860-b29f-7adb96e71519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:09.910722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:09.917913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-14eb380f-ae57-4860-b29f-7adb96e71519 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:09.919622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-14eb380f-ae57-4860-b29f-7adb96e71519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:09.958586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372eab5-8ea8-42de-b78d-a55ed7c45e7e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:09.998773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 50/101] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:10.024220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb380f-ae57-4860-b29f-7adb96e71519 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:10.052247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:10.052921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.053215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.053499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.088448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:10.094150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:10.095480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:10.096269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:10.104107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 52/102] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:10.110051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:10.137721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb380f-ae57-4860-b29f-7adb96e71519 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:10.143886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:10.179245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:10.179860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.180155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.180598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.182116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:10.183637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:10.185647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 51/103] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:10.191541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:10.196312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:10.306418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cc1d0-7c8d-4450-8d98-031e40c6ce1e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:10.343727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873532d-4f71-413b-a2b9-e35b4cf04e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:10.344931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873532d-4f71-413b-a2b9-e35b4cf04e4a None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.347050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873532d-4f71-413b-a2b9-e35b4cf04e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.347050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873532d-4f71-413b-a2b9-e35b4cf04e4a None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.347050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873532d-4f71-413b-a2b9-e35b4cf04e4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:10.347800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873532d-4f71-413b-a2b9-e35b4cf04e4a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:10.355219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c0a17-a9ea-4f62-92f2-d192da10ca6c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:10.396072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798c02c-c2f0-4a72-a5df-816f92f2ba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:10.396714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798c02c-c2f0-4a72-a5df-816f92f2ba0d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.397070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798c02c-c2f0-4a72-a5df-816f92f2ba0d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.397351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798c02c-c2f0-4a72-a5df-816f92f2ba0d None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.398190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798c02c-c2f0-4a72-a5df-816f92f2ba0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:10.398510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798c02c-c2f0-4a72-a5df-816f92f2ba0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:10.399943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798c02c-c2f0-4a72-a5df-816f92f2ba0d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:10.409021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 53/104] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:10.410323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf47d3-2b3a-4cae-8e31-7e436fd49f17 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.410622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf47d3-2b3a-4cae-8e31-7e436fd49f17 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.410911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf47d3-2b3a-4cae-8e31-7e436fd49f17 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.418204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 54/105] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:10.418204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf47d3-2b3a-4cae-8e31-7e436fd49f17 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:10.468407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 52/106] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:41:10 2022] POST /identity/v3/projects => generated 302 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:10.470060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628f643-a32e-4d1b-be15-a7c165b8e074 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.470414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628f643-a32e-4d1b-be15-a7c165b8e074 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.470700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628f643-a32e-4d1b-be15-a7c165b8e074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.511174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:10.518740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5628f643-a32e-4d1b-be15-a7c165b8e074 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:10.520081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5628f643-a32e-4d1b-be15-a7c165b8e074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:10.541164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf47d3-2b3a-4cae-8e31-7e436fd49f17 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:10.582777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:10.583862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.584292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.584582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.585951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:10.587621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:10.589422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 55/107] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:10 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:10.600863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:10.680757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:10.741219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 53/108] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:10.779129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628f643-a32e-4d1b-be15-a7c165b8e074 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:10.786657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5215ff-55b6-4ea9-a071-90071d15ab68 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:10.835346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aeb4f2-ae5c-40f3-8774-f8e3f8f0ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:10.836097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aeb4f2-ae5c-40f3-8774-f8e3f8f0ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.836393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aeb4f2-ae5c-40f3-8774-f8e3f8f0ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.837105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aeb4f2-ae5c-40f3-8774-f8e3f8f0ccd5 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.839162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aeb4f2-ae5c-40f3-8774-f8e3f8f0ccd5 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:10.841111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aeb4f2-ae5c-40f3-8774-f8e3f8f0ccd5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:10.844121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 56/109] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:10 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:10.846187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bbe77-ba51-4284-8c76-d9122f908510 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.846629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bbe77-ba51-4284-8c76-d9122f908510 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.847054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bbe77-ba51-4284-8c76-d9122f908510 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.848741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 57/110] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:10.855006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402381b-bb39-4be7-b0f2-ce0dab6a9f2d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.855787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402381b-bb39-4be7-b0f2-ce0dab6a9f2d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.856217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402381b-bb39-4be7-b0f2-ce0dab6a9f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:10.878926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c402381b-bb39-4be7-b0f2-ce0dab6a9f2d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:10.880304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c402381b-bb39-4be7-b0f2-ce0dab6a9f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:10.889183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:10.919808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628f643-a32e-4d1b-be15-a7c165b8e074 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:10.974923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 58/111] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:10.976943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402381b-bb39-4be7-b0f2-ce0dab6a9f2d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:10.982440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90df49-c549-43a6-af1d-985be90db997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:10.982901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90df49-c549-43a6-af1d-985be90db997 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:10.983241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90df49-c549-43a6-af1d-985be90db997 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:10.983541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90df49-c549-43a6-af1d-985be90db997 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.007179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90df49-c549-43a6-af1d-985be90db997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7a90df49-c549-43a6-af1d-985be90db997 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:11.012683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:11.013545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:11.014316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 54/112] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:11.015272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d4bef-f1a7-45a4-9fd8-506b5ad0afc7 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.019109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d4bef-f1a7-45a4-9fd8-506b5ad0afc7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.019109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d4bef-f1a7-45a4-9fd8-506b5ad0afc7 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.019610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 55/113] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:11.020438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d4bef-f1a7-45a4-9fd8-506b5ad0afc7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.075831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:11.130393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402381b-bb39-4be7-b0f2-ce0dab6a9f2d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:11.144415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d4bef-f1a7-45a4-9fd8-506b5ad0afc7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:11.179386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b195195-cb98-4f3a-89d9-0a1150302094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:11.180662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195195-cb98-4f3a-89d9-0a1150302094 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.181556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195195-cb98-4f3a-89d9-0a1150302094 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.182142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195195-cb98-4f3a-89d9-0a1150302094 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/5bbed68b31dc4a55a7039cbb96dc0c97/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.184495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b195195-cb98-4f3a-89d9-0a1150302094 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b, user_id=5bbed68b31dc4a55a7039cbb96dc0c97, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:11.186152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b195195-cb98-4f3a-89d9-0a1150302094 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:11.186614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:11.187379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.188357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.188707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.200380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:11.206612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:11.207494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:11.208528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:11.209606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 56/114] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:11.210584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.242269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 59/115] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:10 2022] PUT /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/5bbed68b31dc4a55a7039cbb96dc0c97/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:11.243969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff269a-7c38-4d84-8ced-7952859c0fb6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.244466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff269a-7c38-4d84-8ced-7952859c0fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.244912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff269a-7c38-4d84-8ced-7952859c0fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.254494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:11.274802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d9ff269a-7c38-4d84-8ced-7952859c0fb6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:11.276964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d9ff269a-7c38-4d84-8ced-7952859c0fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:11.363010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 60/116] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:11.364074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff269a-7c38-4d84-8ced-7952859c0fb6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.446633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53339cbf-76ea-4f62-897d-01db1ecf1740 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:11.461569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:11.492066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff269a-7c38-4d84-8ced-7952859c0fb6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:11.506727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:11.507566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.507873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.508444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.509427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:11.510099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:11.511591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:11.520707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 57/117] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:11.523609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.532767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:11.538651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a6624-73ce-46c3-a789-50a1a394fcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:11.539297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a6624-73ce-46c3-a789-50a1a394fcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.539682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a6624-73ce-46c3-a789-50a1a394fcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.539970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a6624-73ce-46c3-a789-50a1a394fcf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.540810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a6624-73ce-46c3-a789-50a1a394fcf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:11.541132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a6624-73ce-46c3-a789-50a1a394fcf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:11.552261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a6624-73ce-46c3-a789-50a1a394fcf2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:11.552261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 61/118] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:11.553566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd46b4-a8e0-4638-b8ce-c853f84b97db None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.553566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd46b4-a8e0-4638-b8ce-c853f84b97db None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.553799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd46b4-a8e0-4638-b8ce-c853f84b97db None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.555024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 62/119] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:11.557018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd46b4-a8e0-4638-b8ce-c853f84b97db None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.639513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a818d85-fd6e-4d76-8ae2-0a2954270efa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:11.678321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:11.679260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.679776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.680234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.691177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:11.692782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd46b4-a8e0-4638-b8ce-c853f84b97db None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:11.697514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:11.698380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:11.699220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:11.701374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 58/120] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:11.702296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.735230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:11.744761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a9afc-ba49-4824-a400-f31efd202852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:11.745529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a9afc-ba49-4824-a400-f31efd202852 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.746019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a9afc-ba49-4824-a400-f31efd202852 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.746406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a9afc-ba49-4824-a400-f31efd202852 None admin] PATH_INFO: `/v3/users/d9851b5529994bffa5030a32d33dd608` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.777103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a9afc-ba49-4824-a400-f31efd202852 None admin] RBAC: Authorizing `identity:get_user(user_id=d9851b5529994bffa5030a32d33dd608)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:11.779070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a9afc-ba49-4824-a400-f31efd202852 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:11.782180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 63/121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/users/d9851b5529994bffa5030a32d33dd608 => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:11.783934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78f90-c3f1-474f-9e1e-690bb797414a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.784172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78f90-c3f1-474f-9e1e-690bb797414a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.784455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78f90-c3f1-474f-9e1e-690bb797414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.808795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-eae78f90-c3f1-474f-9e1e-690bb797414a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:11.811286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-eae78f90-c3f1-474f-9e1e-690bb797414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:11.823319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552f78d-01ee-4de8-a44f-0ee3ba29cfb1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:11.864445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:11.865442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.865992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.866495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] PATH_INFO: `/v3/users/3ab27936a00c428f91a5c67560723447` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.868114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] RBAC: Authorizing `identity:get_user(user_id=3ab27936a00c428f91a5c67560723447)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:11.870007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:11.872026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 59/122] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/users/3ab27936a00c428f91a5c67560723447 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:11.873194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.874993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:11.901492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 64/123] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:11.902859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae78f90-c3f1-474f-9e1e-690bb797414a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:11.996283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.003548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8842-aa40-4862-a65c-08e124e5f938 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.021848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae78f90-c3f1-474f-9e1e-690bb797414a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.042751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.043382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.043707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.044086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.044710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:12.045019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.046424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.055629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 60/124] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:12.056458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.066097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.067053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.067562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.068051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.128857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.150341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.152344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.159913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114782d6-3b7c-4252-995b-00605c7bcd1f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.197516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.198231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.198562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.198839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.200063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.201600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.204119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 61/125] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:12.205048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.233974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.273538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 65/126] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:12.274910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.309417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610d98d-486d-425c-bf6a-2be0aabb62f3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.346155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.346806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.347093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.347373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.357921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:12.363931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:12.364821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:12.366163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 62/127] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:12.366824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.370773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.392210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5438-0a80-4dad-899b-a48fc3525782 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.431302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.432362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.432991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.433493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.434965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.436828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.439728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 66/128] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:12.440998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.456529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.473086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba427e-3eea-4c71-a782-2bb194bd63f5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.509938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.510635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.510910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.511197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/3ab27936a00c428f91a5c67560723447/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.513309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b, user_id=3ab27936a00c428f91a5c67560723447, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.514784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.558444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba8585-fdc7-41ef-844a-08b6474da54d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.598716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.600222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.601055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.601594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.603192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.603584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 63/129] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:12 2022] PUT /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/3ab27936a00c428f91a5c67560723447/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:12.604594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.605631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.608310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 67/130] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:12.611158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.639323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.798442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d183-d4f9-4401-aa57-033da943d88d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.818778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7548-cfad-4941-b195-b31a87937ec3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.849564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.850522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.851021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.851493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.852538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:12.853035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.854715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.866200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:12.866866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:12.867157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:12.867440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:12.869835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 64/131] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:12.870709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:12.871681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.873390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:12.876129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 68/132] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:12.877092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:12.887688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.975124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:12.991513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898602-82db-4786-90c6-f56348e8a66c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:12.992857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4b1cc-3b1a-4253-84f6-aaa2e1317884 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:13.030797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18e2f5-ea7f-461c-ba85-b08aee5eadfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:13.031236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:13.032264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18e2f5-ea7f-461c-ba85-b08aee5eadfd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.032807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18e2f5-ea7f-461c-ba85-b08aee5eadfd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.032918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.033258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18e2f5-ea7f-461c-ba85-b08aee5eadfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.033475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.033985 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.034365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18e2f5-ea7f-461c-ba85-b08aee5eadfd None admin] RBAC: Adding query filter params (role.id=85e0cedcfe2043da8a4694814f266cd5, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=d9851b5529994bffa5030a32d33dd608) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:13.034921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18e2f5-ea7f-461c-ba85-b08aee5eadfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:13.035680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:13.036729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18e2f5-ea7f-461c-ba85-b08aee5eadfd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:13.037587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:13.039632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 69/133] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:13.040744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:13.043717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 65/134] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:12 2022] GET /identity/v3/role_assignments?user.id=d9851b5529994bffa5030a32d33dd608&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=85e0cedcfe2043da8a4694814f266cd5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:13.045424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d4f1-4ddd-42a1-adea-c1c27e6de5e8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.045839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d4f1-4ddd-42a1-adea-c1c27e6de5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.046218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d4f1-4ddd-42a1-adea-c1c27e6de5e8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.047756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 66/135] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:13.048649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d4f1-4ddd-42a1-adea-c1c27e6de5e8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:13.144170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:13.164000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1678f5-3bcd-4e86-90c2-590823070429 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:13.182166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d4f1-4ddd-42a1-adea-c1c27e6de5e8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:13.205069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:13.206122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.206582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.207059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/5bbed68b31dc4a55a7039cbb96dc0c97/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.210138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b, user_id=5bbed68b31dc4a55a7039cbb96dc0c97, role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:13.212145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:13.226663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d777c3-9211-4cfb-9110-c949816142a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:13.227331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d777c3-9211-4cfb-9110-c949816142a2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.231737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d777c3-9211-4cfb-9110-c949816142a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.231737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d777c3-9211-4cfb-9110-c949816142a2 None admin] PATH_INFO: `/v3/projects/548ed9cc5ec440269d1066d88287d7fb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.232940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d777c3-9211-4cfb-9110-c949816142a2 None admin] RBAC: Authorizing `identity:get_project(project_id=548ed9cc5ec440269d1066d88287d7fb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:13.235214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d777c3-9211-4cfb-9110-c949816142a2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:13.237986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 67/136] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:13 2022] GET /identity/v3/projects/548ed9cc5ec440269d1066d88287d7fb => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:13.239875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a5536-a7ae-4519-9fa2-88c5bf148e44 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.240475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a5536-a7ae-4519-9fa2-88c5bf148e44 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.241021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a5536-a7ae-4519-9fa2-88c5bf148e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.263427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-601a5536-a7ae-4519-9fa2-88c5bf148e44 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:13.265267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-601a5536-a7ae-4519-9fa2-88c5bf148e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:13.295675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 70/137] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:13 2022] PUT /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/5bbed68b31dc4a55a7039cbb96dc0c97/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:13.301211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:13.342225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:13.410505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 68/138] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:13.448432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a5536-a7ae-4519-9fa2-88c5bf148e44 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:13.499844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f8ede-6cab-4e71-b44d-f1a930a1bdc6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:13.554584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:13.585941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a5536-a7ae-4519-9fa2-88c5bf148e44 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:13.602216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb99c38-7e92-4fd2-8850-a48f6eb580ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:13.602912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb99c38-7e92-4fd2-8850-a48f6eb580ee None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.603202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb99c38-7e92-4fd2-8850-a48f6eb580ee None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.603486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb99c38-7e92-4fd2-8850-a48f6eb580ee None admin] PATH_INFO: `/v3/projects/548ed9cc5ec440269d1066d88287d7fb/users/5bbed68b31dc4a55a7039cbb96dc0c97/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.606347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb99c38-7e92-4fd2-8850-a48f6eb580ee None admin] RBAC: Authorizing `identity:create_grant(project_id=548ed9cc5ec440269d1066d88287d7fb, user_id=5bbed68b31dc4a55a7039cbb96dc0c97, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:13.609377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb99c38-7e92-4fd2-8850-a48f6eb580ee None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:13.635376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:13.636076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.636367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.636703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.638008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:13.639501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:13.641757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 69/139] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:13.647859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:13.683185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 71/140] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:13 2022] PUT /identity/v3/projects/548ed9cc5ec440269d1066d88287d7fb/users/5bbed68b31dc4a55a7039cbb96dc0c97/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:13.737145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe0837-082e-4f20-b1d9-18193917e7a6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.737495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe0837-082e-4f20-b1d9-18193917e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.737834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe0837-082e-4f20-b1d9-18193917e7a6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.739321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 72/141] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:13.744822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9418ef-6c52-4814-885b-0a1986ec868d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.745113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9418ef-6c52-4814-885b-0a1986ec868d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.745398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9418ef-6c52-4814-885b-0a1986ec868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.781136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0f9418ef-6c52-4814-885b-0a1986ec868d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:13.783959 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0f9418ef-6c52-4814-885b-0a1986ec868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:13.832166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46968d37-2256-4c92-bba0-ad57d858d414 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:13.880705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:13.881412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:13.881705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:13.881989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:13.884236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:13.886121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:13.907932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:13.914476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 73/142] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:13.940072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9418ef-6c52-4814-885b-0a1986ec868d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:13.949669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:14.005902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 70/143] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:13 2022] POST /identity/v3/users => generated 283 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:41:14.058960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9418ef-6c52-4814-885b-0a1986ec868d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:14.108406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:14.109096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:14.109390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:14.110278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:14.124640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:14.130515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:14.131353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:14.132159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:14.138985 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 74/144] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:14.140392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:14.284821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f43020-4077-406c-b290-35e008d9dc95 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:14.321807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:14.322449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:14.322799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:14.323082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:14.323856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:14.324164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:14.325675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:14.339488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 71/145] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:14.345997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:14.428029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:14.480955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0202e-84e3-4c1a-8dd6-981c890359c9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:14.525816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:14.526497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:14.526788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:14.527070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:14.528543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:14.530053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:14.532769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 75/146] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:14 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:14.538600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:14.612554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:14.661674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02346c0-2b80-4864-87d6-8dfe841b9020 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:14.697755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:14.698388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:14.698674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:14.698951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:14.772926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:14.774912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:14.835374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:14.882553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 72/147] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:14.888220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:15.002363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b813bf-42c6-49da-9bd1-90a790aabe78 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:15.044815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:15.050657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da80504-d102-4e99-abf4-e7ed9e0f0f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:15.051321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80504-d102-4e99-abf4-e7ed9e0f0f79 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:15.051696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80504-d102-4e99-abf4-e7ed9e0f0f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:15.051976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80504-d102-4e99-abf4-e7ed9e0f0f79 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:15.056234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da80504-d102-4e99-abf4-e7ed9e0f0f79 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:15.057712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da80504-d102-4e99-abf4-e7ed9e0f0f79 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:15.059390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 76/148] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:14 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:15.065322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:15.189182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f57ab-954a-4845-8e8a-db9022fcd5a1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:15.227631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5617f-3b32-4d44-bb19-6953103eacc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:15.228268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5617f-3b32-4d44-bb19-6953103eacc6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:15.228549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5617f-3b32-4d44-bb19-6953103eacc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:15.228829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5617f-3b32-4d44-bb19-6953103eacc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:15.229631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5617f-3b32-4d44-bb19-6953103eacc6 None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=5bbed68b31dc4a55a7039cbb96dc0c97) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:15.229945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5617f-3b32-4d44-bb19-6953103eacc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:15.231410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5617f-3b32-4d44-bb19-6953103eacc6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:15.239311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 73/149] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:15 2022] GET /identity/v3/role_assignments?user.id=5bbed68b31dc4a55a7039cbb96dc0c97&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:16.103135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9021f-5cec-46d8-a851-b9659b880963 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.103640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9021f-5cec-46d8-a851-b9659b880963 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.103640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9021f-5cec-46d8-a851-b9659b880963 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.105002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 77/150] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:16.110887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2226b75-8a56-47c3-a9ad-fbef76440528 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.111326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2226b75-8a56-47c3-a9ad-fbef76440528 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.111426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2226b75-8a56-47c3-a9ad-fbef76440528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.141097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b2226b75-8a56-47c3-a9ad-fbef76440528 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:16.142445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b2226b75-8a56-47c3-a9ad-fbef76440528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:16.162113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:16.234207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 74/151] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:16.269548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9021f-5cec-46d8-a851-b9659b880963 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:16.319910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05629bb3-db54-4c42-ba30-95b6277bd570 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.320539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05629bb3-db54-4c42-ba30-95b6277bd570 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.321027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05629bb3-db54-4c42-ba30-95b6277bd570 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.323062 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 75/152] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:41:16.330313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564613ea-70fd-4cb7-bb48-b42b4b994283 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.331195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564613ea-70fd-4cb7-bb48-b42b4b994283 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.331721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564613ea-70fd-4cb7-bb48-b42b4b994283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.355230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-564613ea-70fd-4cb7-bb48-b42b4b994283 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:16.356948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-564613ea-70fd-4cb7-bb48-b42b4b994283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:16.426414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9021f-5cec-46d8-a851-b9659b880963 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:16.484399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 76/153] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:16.487163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753433a-b1fc-4893-82ff-e4faf668364b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.488188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753433a-b1fc-4893-82ff-e4faf668364b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.488470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753433a-b1fc-4893-82ff-e4faf668364b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.490785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 77/154] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:16.495259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99086d-8abc-48aa-bcba-5d2dbb7959cd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.495558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99086d-8abc-48aa-bcba-5d2dbb7959cd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.495867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99086d-8abc-48aa-bcba-5d2dbb7959cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.496994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:16.497703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.497981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.498253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.512010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:16.518166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:16.519053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:16.520239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:16.522304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 78/155] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:16.528367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:16.528495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4c99086d-8abc-48aa-bcba-5d2dbb7959cd None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:16.529931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4c99086d-8abc-48aa-bcba-5d2dbb7959cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:16.596572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:16.635845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 78/156] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:16.635845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99086d-8abc-48aa-bcba-5d2dbb7959cd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:16.651676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702de4d-1cda-46f8-954d-ea35c796cafe None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:16.654527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:16.690017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:16.691117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.691414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.691727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.692467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:16.692783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:16.694484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:16.705179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 79/157] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:16.706178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:16.838259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb10b18-ff01-410e-a769-221ecb47e046 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:16.865580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:16.873760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99086d-8abc-48aa-bcba-5d2dbb7959cd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:16.881419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:16.882082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.882374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.882647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.910179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:16.919276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:16.920240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:16.921036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:16.924344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 80/158] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:16.925170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:16.949418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:16.950077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.950362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.950638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.970434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:16.976600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:16.977535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:16.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 79/159] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:16.979035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.071803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:17.072458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41585a-5a21-4829-b0e2-416cc1ade758 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.105835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3cdea-907e-424a-a4f1-7d8d7b6baa77 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.110951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.112208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.112697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.113169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] PATH_INFO: `/v3/users/d9851b5529994bffa5030a32d33dd608` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.114603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] RBAC: Authorizing `identity:get_user(user_id=d9851b5529994bffa5030a32d33dd608)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.116582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.118420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 81/160] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:16 2022] GET /identity/v3/users/d9851b5529994bffa5030a32d33dd608 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:17.119381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.142929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.143623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.143862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.144193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.144921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:17.145284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.146872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.155750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 80/161] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:17.156668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.180356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:17.236937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7ea42-3039-4f45-9ec2-ecd122ac0a47 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.264674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16e80-d7a5-4d0a-aed9-e1ae892179be None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.284538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.285513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.285998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.286474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.287438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:17.287970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.289575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.308189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 82/162] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:17.310517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.331924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:17.344734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.345551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.345847 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.346131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.350521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.352291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.354070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 81/163] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:17 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:17.357310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.454048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b012b21-4765-4fbd-918b-e6bb63fb4c64 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.480032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57de9a-c86e-4c51-bb13-84ba59839a9e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.486464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:17.519805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5aa90-d67d-47ed-8f08-4a8ff6c6259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.520890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5aa90-d67d-47ed-8f08-4a8ff6c6259e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.521376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5aa90-d67d-47ed-8f08-4a8ff6c6259e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.521850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5aa90-d67d-47ed-8f08-4a8ff6c6259e None admin] PATH_INFO: `/v3/users/3ab27936a00c428f91a5c67560723447` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.523275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5aa90-d67d-47ed-8f08-4a8ff6c6259e None admin] RBAC: Authorizing `identity:get_user(user_id=3ab27936a00c428f91a5c67560723447)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.525263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.525516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5aa90-d67d-47ed-8f08-4a8ff6c6259e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.525924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.526214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.526497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.528368 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 82/164] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:17 2022] GET /identity/v3/users/3ab27936a00c428f91a5c67560723447 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:17.529887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a25b8-47f2-4b2f-b622-080aabd0a5f4 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.530390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a25b8-47f2-4b2f-b622-080aabd0a5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.530871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a25b8-47f2-4b2f-b622-080aabd0a5f4 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.532274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.532576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 83/165] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:17.535572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.535572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 83/166] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:17 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:17.536188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.536574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a25b8-47f2-4b2f-b622-080aabd0a5f4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.627832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:17.659332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a25b8-47f2-4b2f-b622-080aabd0a5f4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.696483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505eface-d568-44a4-81bf-24f581809a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.697113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505eface-d568-44a4-81bf-24f581809a9b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.697403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505eface-d568-44a4-81bf-24f581809a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.697684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505eface-d568-44a4-81bf-24f581809a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.752085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc351d50-25f8-46f1-8bde-c6eeb079c0de None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:17.779564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505eface-d568-44a4-81bf-24f581809a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.781333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505eface-d568-44a4-81bf-24f581809a9b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.807098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:17.807930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.808221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.808504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/d9851b5529994bffa5030a32d33dd608/roles/85e0cedcfe2043da8a4694814f266cd5` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.810593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b, user_id=d9851b5529994bffa5030a32d33dd608, role_id=85e0cedcfe2043da8a4694814f266cd5)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:17.812210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:17.852377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:17.883680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 84/167] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:17 2022] PUT /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/users/d9851b5529994bffa5030a32d33dd608/roles/85e0cedcfe2043da8a4694814f266cd5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:17.884644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:17.891272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 84/168] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:17.892735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a1e31-61f0-4b30-9c01-4bb367bd22d2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:17.893041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a1e31-61f0-4b30-9c01-4bb367bd22d2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:17.893321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a1e31-61f0-4b30-9c01-4bb367bd22d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:17.917581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-836a1e31-61f0-4b30-9c01-4bb367bd22d2 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:17.918947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-836a1e31-61f0-4b30-9c01-4bb367bd22d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:17.940702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:18.017605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd495-9e18-4fea-9116-f01748d5a6e3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:18.019778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 85/169] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:18.021262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09845c8c-5fe7-49ee-bcdd-798d8cea1c8e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.021664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09845c8c-5fe7-49ee-bcdd-798d8cea1c8e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.022042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09845c8c-5fe7-49ee-bcdd-798d8cea1c8e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.023881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 86/170] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:18.024966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09845c8c-5fe7-49ee-bcdd-798d8cea1c8e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:18.059401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:18.060298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.060684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.061079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.155419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:18.161460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09845c8c-5fe7-49ee-bcdd-798d8cea1c8e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:18.168939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:18.170495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:18.204718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59219b7a-5c3b-4c76-a41d-2a0f99a95f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:18.205391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59219b7a-5c3b-4c76-a41d-2a0f99a95f25 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.205678 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59219b7a-5c3b-4c76-a41d-2a0f99a95f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.206106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59219b7a-5c3b-4c76-a41d-2a0f99a95f25 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.207699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59219b7a-5c3b-4c76-a41d-2a0f99a95f25 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:18.209617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59219b7a-5c3b-4c76-a41d-2a0f99a95f25 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:18.211558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 87/171] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:18.213498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b1c55-a82b-40d1-b1ba-e4eea35b5469 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.213795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b1c55-a82b-40d1-b1ba-e4eea35b5469 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.214079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b1c55-a82b-40d1-b1ba-e4eea35b5469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.246323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bf7b1c55-a82b-40d1-b1ba-e4eea35b5469 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:18.256221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bf7b1c55-a82b-40d1-b1ba-e4eea35b5469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:18.257186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:18.339833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 85/172] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:18.340742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:18.409901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 88/173] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:18.411112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b1c55-a82b-40d1-b1ba-e4eea35b5469 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:18.492216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd5377-7bca-4667-af91-1a905378e838 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:18.542986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:18.543760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.544550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.544550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.555173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:18.560764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b1c55-a82b-40d1-b1ba-e4eea35b5469 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:18.563162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:18.566827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:18.580166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:18.581221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:18.582043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 86/174] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:18.582737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:18.611833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:18.612468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.612755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.613034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.613841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=3ab27936a00c428f91a5c67560723447) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:18.614160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:18.615653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:18.628151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 89/175] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/role_assignments?user.id=3ab27936a00c428f91a5c67560723447&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:18.630333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:18.757715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51401bfb-c95b-44e7-93d2-c3c142383888 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:18.798732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:18.799399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.799736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.800018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.801292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:18.802942 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:18.804706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 87/176] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:18.805189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d927f78-63f5-48b5-9085-e33ed8cdc7eb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:18.805500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:18.854651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a889bf-6c22-4cc6-888d-7dc7bfc91740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:18.855309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a889bf-6c22-4cc6-888d-7dc7bfc91740 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.856217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a889bf-6c22-4cc6-888d-7dc7bfc91740 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.856414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a889bf-6c22-4cc6-888d-7dc7bfc91740 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.872523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a889bf-6c22-4cc6-888d-7dc7bfc91740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d8a889bf-6c22-4cc6-888d-7dc7bfc91740 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:18.880176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:18.881121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:18.881928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 90/177] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:18.883457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d537c-ea28-49f5-afcf-bd2f9d466521 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.883457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d537c-ea28-49f5-afcf-bd2f9d466521 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.883457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d537c-ea28-49f5-afcf-bd2f9d466521 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.884338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:18.891452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 91/178] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:18.891452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faeeee-b539-4257-8814-5a0fcc8dc9b7 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.891452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faeeee-b539-4257-8814-5a0fcc8dc9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.891452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faeeee-b539-4257-8814-5a0fcc8dc9b7 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.891452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 92/179] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:18.891994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faeeee-b539-4257-8814-5a0fcc8dc9b7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:18.927882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:18.928215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec6cdc-eb0b-40bb-8972-bfbe4800bcca None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:18.971087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:18.971759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:18.972046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:18.972386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:18.973107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:18.973423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:18.974815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:18.983180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 88/180] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:18.983994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.031330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faeeee-b539-4257-8814-5a0fcc8dc9b7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.067035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395c7e8-ae66-43a9-a88a-0837e62fb7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.067686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395c7e8-ae66-43a9-a88a-0837e62fb7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.067973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395c7e8-ae66-43a9-a88a-0837e62fb7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.068247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395c7e8-ae66-43a9-a88a-0837e62fb7e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.069043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395c7e8-ae66-43a9-a88a-0837e62fb7e9 None admin] RBAC: Adding query filter params (role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=5bbed68b31dc4a55a7039cbb96dc0c97) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:19.069571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395c7e8-ae66-43a9-a88a-0837e62fb7e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:19.070809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395c7e8-ae66-43a9-a88a-0837e62fb7e9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:19.077422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 93/181] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/role_assignments?user.id=5bbed68b31dc4a55a7039cbb96dc0c97&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:19.078402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac229d-2c49-4ead-a781-3db80d2520d4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.078785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac229d-2c49-4ead-a781-3db80d2520d4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.079069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac229d-2c49-4ead-a781-3db80d2520d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.098748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-75ac229d-2c49-4ead-a781-3db80d2520d4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:19.100120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-75ac229d-2c49-4ead-a781-3db80d2520d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:19.119269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b4562-42d4-4360-97a4-6ac70c3883e7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.168396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.178246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.179936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.180488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.180924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.181670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 94/182] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:19.187228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bbaad-3e01-4a0d-8060-624e4c4c4f95 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.187228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bbaad-3e01-4a0d-8060-624e4c4c4f95 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.187228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bbaad-3e01-4a0d-8060-624e4c4c4f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.189969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:19.189969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:19.190549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:19.201549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 89/183] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:19.204715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.223837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0a7bbaad-3e01-4a0d-8060-624e4c4c4f95 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:19.225319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0a7bbaad-3e01-4a0d-8060-624e4c4c4f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:19.237192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.319395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 95/184] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:19.320702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bbaad-3e01-4a0d-8060-624e4c4c4f95 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.350069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef3d5-838f-4fbc-87fd-cb66ef513890 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.388440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.389088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.389374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.389647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] PATH_INFO: `/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.391807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbed68b31dc4a55a7039cbb96dc0c97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:19.393271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:19.395166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 90/185] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:19 2022] GET /identity/v3/users/5bbed68b31dc4a55a7039cbb96dc0c97 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:19.395922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.433597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bbaad-3e01-4a0d-8060-624e4c4c4f95 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.459160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.468994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.469660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.469951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.470231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] PATH_INFO: `/v3/users/a695db67d910450f8fabbfcec01c8327` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.483851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] RBAC: Authorizing `identity:get_user(user_id=a695db67d910450f8fabbfcec01c8327)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:19.485369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:19.491318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 96/186] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:19 2022] GET /identity/v3/users/a695db67d910450f8fabbfcec01c8327 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:19.491318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.491318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.504499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e86e2a-c3d7-49f6-b7f2-3df8b74531b5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.543873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.544542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.544811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.545515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.557327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:19.562378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:19.564508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 91/187] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:19.565483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.603425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bcd04-39d4-489a-acc9-39e4b73477f3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.648800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.649464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.649895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.650185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.652013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:19.653924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:19.660182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 97/188] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:19.660182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.668340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67aa9d-f948-4920-82fe-3f0fce2441ec None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.690560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.704634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.705440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.705737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.706018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.741015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.781225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:19.781588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3c71b-b542-4a17-9d3c-ba387ea3eab1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:19.782720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:19.821488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:19.822117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:19.822404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:19.822679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:19.893992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 92/189] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:19.893992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:19.898289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:19.899835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:19.913897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.970313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:19.998239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006468f3-63b5-4820-92f7-69825438a7b1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.003124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 98/190] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:20.004130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.034941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.035860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.036303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.036734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.037452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:20.037758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.039142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:20.048265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 93/191] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:20.048880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.118758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263920f-ee6b-46a9-8617-c9adb53b8620 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.143284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:20.154482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.155112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.155498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.157176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.157176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.158491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:20.162877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc3676-c280-454d-a680-5ffc1650b4b8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.198280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.198955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.199372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.199685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] PATH_INFO: `/v3/projects/548ed9cc5ec440269d1066d88287d7fb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.201061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] RBAC: Authorizing `identity:get_project(project_id=548ed9cc5ec440269d1066d88287d7fb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.202512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:20.204382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 94/192] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/projects/548ed9cc5ec440269d1066d88287d7fb => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:20.205142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.258861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 99/193] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:20 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:20.259654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.306411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:20.312952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c90232-5b60-4d97-964e-7f2e12775da5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.352181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98130a61-c489-4d4c-a666-16b83ff8ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.352841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130a61-c489-4d4c-a666-16b83ff8ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.353125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130a61-c489-4d4c-a666-16b83ff8ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.353396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130a61-c489-4d4c-a666-16b83ff8ebf7 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.354593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130a61-c489-4d4c-a666-16b83ff8ebf7 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.356126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130a61-c489-4d4c-a666-16b83ff8ebf7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:20.357851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 95/194] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:20.359145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9534d-ae51-4a92-849a-e15e6a9f84ad None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.359960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9534d-ae51-4a92-849a-e15e6a9f84ad None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.360343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9534d-ae51-4a92-849a-e15e6a9f84ad None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.366048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 96/195] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:20.366048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9534d-ae51-4a92-849a-e15e6a9f84ad None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.389082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:20.399793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ace918-a687-44d2-b679-a0d1ff7eb9cf None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.441532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.442204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.442474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.442752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] PATH_INFO: `/v3/users/a695db67d910450f8fabbfcec01c8327` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.444018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] RBAC: Authorizing `identity:get_user(user_id=a695db67d910450f8fabbfcec01c8327)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.445484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:20.447249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 100/196] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/users/a695db67d910450f8fabbfcec01c8327 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:20.447874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.488471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9534d-ae51-4a92-849a-e15e6a9f84ad None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.536490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc998cd-333c-413f-a675-b48baa581505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.537150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc998cd-333c-413f-a675-b48baa581505 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.537507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc998cd-333c-413f-a675-b48baa581505 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.537876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc998cd-333c-413f-a675-b48baa581505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.538742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc998cd-333c-413f-a675-b48baa581505 None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=548ed9cc5ec440269d1066d88287d7fb, user.id=5bbed68b31dc4a55a7039cbb96dc0c97) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:20.539131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc998cd-333c-413f-a675-b48baa581505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.540817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc998cd-333c-413f-a675-b48baa581505 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:20.549998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 97/197] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/role_assignments?user.id=5bbed68b31dc4a55a7039cbb96dc0c97&scope.project.id=548ed9cc5ec440269d1066d88287d7fb&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:20.552282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133012f-458c-49a6-b0c0-797e1e279905 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.552576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133012f-458c-49a6-b0c0-797e1e279905 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.552860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133012f-458c-49a6-b0c0-797e1e279905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.575909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:20.576905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f133012f-458c-49a6-b0c0-797e1e279905 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:20.579001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f133012f-458c-49a6-b0c0-797e1e279905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:20.585462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a496e3-9fea-4b49-bdb5-c15203dab129 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.639654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.640521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.640813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.641086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.642410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=a695db67d910450f8fabbfcec01c8327) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:20.642998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.644704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:20.652642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 101/198] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/role_assignments?user.id=a695db67d910450f8fabbfcec01c8327&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:20.654315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.667600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 98/199] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:20.695028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133012f-458c-49a6-b0c0-797e1e279905 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.715524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:20.794594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48063-a8e6-4463-996f-d938309be40c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.821517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133012f-458c-49a6-b0c0-797e1e279905 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:20.845752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.846842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.847343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.847881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.875962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:20.879099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:20.879672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:20.882047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:20.895093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:20.904200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:20.912251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:20.913204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:20.914001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:20.922738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 99/200] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:20.928790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:20.976734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:20.978355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:21.026098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:21.071085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248af4d-1f00-4e20-b4c7-ead39c9cde49 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:21.128168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:21.129891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:21.131488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:21.131996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:21.133141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:21.134626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:21.136310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:21.141844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 102/201] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:21.148869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:21.154994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 100/202] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:21.161505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:21.246566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:21.293640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49df67-c37f-4082-b1e5-7424ad88a2db None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:21.334905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:21.335864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:21.336735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:21.337187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:21.338567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:21.340235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:21.342671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 103/203] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:21 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:21.349538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:21.385016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4659894-97a7-4b35-b894-7fe665be294a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:21.444869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:21.445523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:21.445809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:21.446088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] PATH_INFO: `/v3/users/d9851b5529994bffa5030a32d33dd608` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:21.448482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] RBAC: Authorizing `identity:get_user(user_id=d9851b5529994bffa5030a32d33dd608)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:21.450686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:21.452347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 101/204] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:21 2022] GET /identity/v3/users/d9851b5529994bffa5030a32d33dd608 => generated 289 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:21.460369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:21.479886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:21.523870 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800db96-845b-4b47-8b1d-48c6f697c824 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:21.571880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:21.588536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:21.589098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:21.589384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:21.589666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:21.590474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] RBAC: Adding query filter params (role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=a695db67d910450f8fabbfcec01c8327) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:21.595172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:21.596672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:21.608690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 104/205] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:21 2022] GET /identity/v3/role_assignments?user.id=a695db67d910450f8fabbfcec01c8327&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:21.649565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e67dc-82dc-4062-ae75-55e6ea6e5e82 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:21.708269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:21.708922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:21.709209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:21.709489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:21.824637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:21.826151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:21.829740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:21.968026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 102/206] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:21.973481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:22.085238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850e35a-a65d-4caf-a94d-70afebfb91dc None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:22.120452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12d956-3b2c-4fe9-ac8c-a68f26d3bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:22.121111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12d956-3b2c-4fe9-ac8c-a68f26d3bba8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:22.121399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12d956-3b2c-4fe9-ac8c-a68f26d3bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:22.121683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12d956-3b2c-4fe9-ac8c-a68f26d3bba8 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:22.122877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12d956-3b2c-4fe9-ac8c-a68f26d3bba8 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:22.124383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12d956-3b2c-4fe9-ac8c-a68f26d3bba8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:22.127024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 105/207] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:21 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:22.133072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:22.244604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216456b3-927e-46e1-bd68-b227a04f5264 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:22.246093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:22.279746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7244e-0cf4-4a43-9551-26b5e1357d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:22.280381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7244e-0cf4-4a43-9551-26b5e1357d2e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:22.280665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7244e-0cf4-4a43-9551-26b5e1357d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:22.281577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7244e-0cf4-4a43-9551-26b5e1357d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:22.282296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7244e-0cf4-4a43-9551-26b5e1357d2e None admin] RBAC: Adding query filter params (role.id=85e0cedcfe2043da8a4694814f266cd5, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b, user.id=d9851b5529994bffa5030a32d33dd608) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:22.283100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7244e-0cf4-4a43-9551-26b5e1357d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:22.284933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7244e-0cf4-4a43-9551-26b5e1357d2e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:22.291087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 103/208] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:22 2022] GET /identity/v3/role_assignments?user.id=d9851b5529994bffa5030a32d33dd608&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=85e0cedcfe2043da8a4694814f266cd5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:22.703925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9eca-7f22-4c60-90bf-e31c2f7e0af8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:22.704436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9eca-7f22-4c60-90bf-e31c2f7e0af8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:22.704539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9eca-7f22-4c60-90bf-e31c2f7e0af8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:22.706145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 106/209] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:22.711571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd631e0-2e82-4f4d-9e3a-1906caf7e839 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:22.711975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd631e0-2e82-4f4d-9e3a-1906caf7e839 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:22.712155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd631e0-2e82-4f4d-9e3a-1906caf7e839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:22.731445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3fd631e0-2e82-4f4d-9e3a-1906caf7e839 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:22.732707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3fd631e0-2e82-4f4d-9e3a-1906caf7e839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:22.816254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 104/210] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:22.843955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe9eca-7f22-4c60-90bf-e31c2f7e0af8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:22.912568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:22.960097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe9eca-7f22-4c60-90bf-e31c2f7e0af8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.000673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d70e5-0432-4094-8298-f96a6f44a4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.001322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d70e5-0432-4094-8298-f96a6f44a4db None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.001606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d70e5-0432-4094-8298-f96a6f44a4db None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.001882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d70e5-0432-4094-8298-f96a6f44a4db None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.012859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d70e5-0432-4094-8298-f96a6f44a4db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2e4d70e5-0432-4094-8298-f96a6f44a4db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:23.019023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:23.019920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:23.020727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 107/211] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:23.026098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd631e0-2e82-4f4d-9e3a-1906caf7e839 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.028741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38f98f-ebef-4ef6-b782-3f9a3c4abaa7 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.029035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38f98f-ebef-4ef6-b782-3f9a3c4abaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.029326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38f98f-ebef-4ef6-b782-3f9a3c4abaa7 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.031490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 108/212] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:23.036573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b855-5ac3-491c-bab6-e5b33e290977 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.037114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b855-5ac3-491c-bab6-e5b33e290977 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.037703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b855-5ac3-491c-bab6-e5b33e290977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.057167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0968b855-5ac3-491c-bab6-e5b33e290977 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:23.058752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0968b855-5ac3-491c-bab6-e5b33e290977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:23.126523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:23.132534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd631e0-2e82-4f4d-9e3a-1906caf7e839 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.136875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 109/213] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:23.162307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b855-5ac3-491c-bab6-e5b33e290977 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.177064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.177707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.177995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.178272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.179010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:23.179359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:23.180757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:23.195147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 105/214] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:23.196369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.226709 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:23.279465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b855-5ac3-491c-bab6-e5b33e290977 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.305137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c34c3-33d7-4364-b797-3aa0d61825bb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.317101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0740502-b2ca-4da3-9bc4-95e99e39b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.317750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0740502-b2ca-4da3-9bc4-95e99e39b828 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.318041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0740502-b2ca-4da3-9bc4-95e99e39b828 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.318321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0740502-b2ca-4da3-9bc4-95e99e39b828 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.319477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0740502-b2ca-4da3-9bc4-95e99e39b828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:23.321045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0740502-b2ca-4da3-9bc4-95e99e39b828 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:23.322676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 110/215] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:23.323973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba84f4c-8dff-4ac6-9c4d-9835c25d6c1b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.324267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba84f4c-8dff-4ac6-9c4d-9835c25d6c1b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.324546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba84f4c-8dff-4ac6-9c4d-9835c25d6c1b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.325891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 111/216] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:23.329245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba84f4c-8dff-4ac6-9c4d-9835c25d6c1b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.348504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3e697-57c1-4682-8bf8-9c17c99d4e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.349154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3e697-57c1-4682-8bf8-9c17c99d4e29 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.349463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3e697-57c1-4682-8bf8-9c17c99d4e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.349754 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3e697-57c1-4682-8bf8-9c17c99d4e29 None admin] PATH_INFO: `/v3/users/9f474fc13a4b4397b1335234d3eb88c2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.364233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3e697-57c1-4682-8bf8-9c17c99d4e29 None admin] RBAC: Authorizing `identity:get_user(user_id=9f474fc13a4b4397b1335234d3eb88c2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:23.365685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3e697-57c1-4682-8bf8-9c17c99d4e29 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:23.367506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 106/217] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:23 2022] GET /identity/v3/users/9f474fc13a4b4397b1335234d3eb88c2 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:23.372871 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b42fc6-03e0-4935-b0fa-1b6e9f8128e3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.373166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b42fc6-03e0-4935-b0fa-1b6e9f8128e3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.373451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b42fc6-03e0-4935-b0fa-1b6e9f8128e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.392702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-55b42fc6-03e0-4935-b0fa-1b6e9f8128e3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:23.393889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-55b42fc6-03e0-4935-b0fa-1b6e9f8128e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:23.404779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:23.451359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba84f4c-8dff-4ac6-9c4d-9835c25d6c1b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.463228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 107/218] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:23.467772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b42fc6-03e0-4935-b0fa-1b6e9f8128e3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.481126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:23.501750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.502382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.502670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.502949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.503735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:23.505723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:23.571161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b42fc6-03e0-4935-b0fa-1b6e9f8128e3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.579217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 112/219] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:23 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:23.587584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.615132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.615918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.616321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.616715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.638648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:23.724628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:23.725256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ca55-4e4a-4726-a63a-5716a7be6bb2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.728547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:23.773175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.773851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.774118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.774394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.788275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:23.795144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:23.796142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:23.796995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:23.797719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 113/220] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:23.804446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.845939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:23.854898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 108/221] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:23.861059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:23.918397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:23.938258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a051a34-a91d-49ae-8992-62123cd269e7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.978909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:23.979598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:23.979877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:23.980378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:23.980880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:23.981192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:23.982601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:23.987217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cff5c-a70b-4f81-b297-01f217112956 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:23.993002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 114/222] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:24.003845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.033667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.034746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.035496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.036224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.037742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:24.040021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:24.042649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 109/223] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:23 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:24.049168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.119335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f4530-a011-4d64-994d-c51285b010fa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:24.135565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:24.155399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6028969-9f59-4274-a08c-32f139443826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.156123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6028969-9f59-4274-a08c-32f139443826 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.156453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6028969-9f59-4274-a08c-32f139443826 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.156745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6028969-9f59-4274-a08c-32f139443826 None admin] PATH_INFO: `/v3/users/a695db67d910450f8fabbfcec01c8327` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.157959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6028969-9f59-4274-a08c-32f139443826 None admin] RBAC: Authorizing `identity:get_user(user_id=a695db67d910450f8fabbfcec01c8327)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:24.159440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6028969-9f59-4274-a08c-32f139443826 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:24.159576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aafab3-2543-4b57-b97b-35cb36a0d33e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:24.161468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 115/224] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:24 2022] GET /identity/v3/users/a695db67d910450f8fabbfcec01c8327 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:24.162662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d90ad-c389-476e-8c2a-dd28018e1c01 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.162956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d90ad-c389-476e-8c2a-dd28018e1c01 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.163442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d90ad-c389-476e-8c2a-dd28018e1c01 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.164742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 116/225] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:24.170280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd61de-e95e-459c-9df5-e0a4c125a378 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.170581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd61de-e95e-459c-9df5-e0a4c125a378 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.170849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd61de-e95e-459c-9df5-e0a4c125a378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.179946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:24.190425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8bbd61de-e95e-459c-9df5-e0a4c125a378 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:24.192382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8bbd61de-e95e-459c-9df5-e0a4c125a378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:24.201546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.202258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.202650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.203479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.204371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=9f474fc13a4b4397b1335234d3eb88c2) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:24.204749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:24.206254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:24.212015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 110/226] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:24 2022] GET /identity/v3/role_assignments?user.id=9f474fc13a4b4397b1335234d3eb88c2&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:24.220843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.294993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 117/227] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:24.320945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd61de-e95e-459c-9df5-e0a4c125a378 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.437170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:24.444353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca7614-f851-4ad2-9e83-6782d0bd6957 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:24.455566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd61de-e95e-459c-9df5-e0a4c125a378 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:24.489190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.489992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.490280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.490558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.502629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:24.508059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:24.508934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:24.509723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 118/228] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:24.516270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.526674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.527647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.528252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.528739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.530286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:24.532701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:24.535150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 111/229] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:24 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:24.540578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.631397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaf189-f800-4394-9e48-37fd907ea086 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:24.632920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:24.673229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.673872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.674162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.674448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.675185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:24.675495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:24.676968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:24.685849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 119/230] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:24.691480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.717534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:24.736107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644c6d5-f232-4b75-bc53-1a77d7c426e1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:24.802747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.803434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.803746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.804031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/a695db67d910450f8fabbfcec01c8327/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.812789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb040a5048fb4656b0af59ad909a2338, user_id=a695db67d910450f8fabbfcec01c8327, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:24.814309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:24.843888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abcca-e87b-4fd1-b9fa-101bd6130c8c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:24.879422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:24.880221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:24.880443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:24.880723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] PATH_INFO: `/v3/users/a695db67d910450f8fabbfcec01c8327` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:24.891141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] RBAC: Authorizing `identity:get_user(user_id=a695db67d910450f8fabbfcec01c8327)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:24.891141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:24.891141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 120/231] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:24 2022] GET /identity/v3/users/a695db67d910450f8fabbfcec01c8327 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:24.891141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:24.950866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 112/232] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:24 2022] PUT /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/a695db67d910450f8fabbfcec01c8327/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:24.988349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:25.008569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57474d1-574e-4244-9b02-0c852719d524 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:25.044567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f979914-aec5-4dcf-992a-8826b49b3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:25.045312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f979914-aec5-4dcf-992a-8826b49b3831 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:25.045606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f979914-aec5-4dcf-992a-8826b49b3831 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:25.045885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f979914-aec5-4dcf-992a-8826b49b3831 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:25.047682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f979914-aec5-4dcf-992a-8826b49b3831 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:25.049189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f979914-aec5-4dcf-992a-8826b49b3831 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:25.051182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 121/233] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:24 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:25.057063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:25.203025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5ed89-fdc8-4854-8a92-507d25108594 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:25.212603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:25.240641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3f0d8-7eed-4ee4-941d-6b7521b6e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:25.241334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f0d8-7eed-4ee4-941d-6b7521b6e89a None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:25.241626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f0d8-7eed-4ee4-941d-6b7521b6e89a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:25.241913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f0d8-7eed-4ee4-941d-6b7521b6e89a None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/a695db67d910450f8fabbfcec01c8327/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:25.244168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3f0d8-7eed-4ee4-941d-6b7521b6e89a None admin] RBAC: Authorizing `identity:create_grant(project_id=eb040a5048fb4656b0af59ad909a2338, user_id=a695db67d910450f8fabbfcec01c8327, role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:25.245619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3f0d8-7eed-4ee4-941d-6b7521b6e89a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:25.346139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 113/234] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:25 2022] PUT /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/a695db67d910450f8fabbfcec01c8327/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:26.115519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe050e-be6c-4d59-8e7d-21d86584e76c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.116080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe050e-be6c-4d59-8e7d-21d86584e76c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.116080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe050e-be6c-4d59-8e7d-21d86584e76c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.117444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 122/235] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:26.122762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b78bfe-41b0-4205-bcf8-5599b044e179 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.123187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b78bfe-41b0-4205-bcf8-5599b044e179 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.123508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b78bfe-41b0-4205-bcf8-5599b044e179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.143286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b4b78bfe-41b0-4205-bcf8-5599b044e179 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:26.144668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b4b78bfe-41b0-4205-bcf8-5599b044e179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:26.159911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b2f72-c528-44dc-aee8-3a4f53f351f0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.160224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b2f72-c528-44dc-aee8-3a4f53f351f0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.160461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b2f72-c528-44dc-aee8-3a4f53f351f0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.161818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 123/236] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:41:26.168667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120c2cb-dbca-4129-92a9-8f2a96812c6e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.168820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120c2cb-dbca-4129-92a9-8f2a96812c6e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.169040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120c2cb-dbca-4129-92a9-8f2a96812c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.188854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c120c2cb-dbca-4129-92a9-8f2a96812c6e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:26.190704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c120c2cb-dbca-4129-92a9-8f2a96812c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:26.247023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 114/237] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:26.292915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b78bfe-41b0-4205-bcf8-5599b044e179 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:26.306523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 124/238] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:26.371998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120c2cb-dbca-4129-92a9-8f2a96812c6e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:26.444977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:26.453797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b78bfe-41b0-4205-bcf8-5599b044e179 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:26.500809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:26.501471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.501758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.502035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.512953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:26.519151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:26.520077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:26.520887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 115/239] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:26.529026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:26.547684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:26.583306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120c2cb-dbca-4129-92a9-8f2a96812c6e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:26.622437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:26.623276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.623574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.623854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.626092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:26.627767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:26.634260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 125/240] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:26.636497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:26.683578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95b5d3-7fbb-4aaa-8f5e-acfbe41585a6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:26.719486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7876b-572b-43d9-8c11-ef2782245167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:26.720146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7876b-572b-43d9-8c11-ef2782245167 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.720432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7876b-572b-43d9-8c11-ef2782245167 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.720829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7876b-572b-43d9-8c11-ef2782245167 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.721670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7876b-572b-43d9-8c11-ef2782245167 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:26.721981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7876b-572b-43d9-8c11-ef2782245167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:26.723370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7876b-572b-43d9-8c11-ef2782245167 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:26.732583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 116/241] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:26.733635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7286d-dccd-4e2d-a9b4-7a650f5e421a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.733928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7286d-dccd-4e2d-a9b4-7a650f5e421a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7286d-dccd-4e2d-a9b4-7a650f5e421a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.735682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 117/242] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:26.740692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7286d-dccd-4e2d-a9b4-7a650f5e421a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:26.758062 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d61f1-788e-4d15-a4c0-cb0df44f553b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:26.762562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:26.802060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfadf8-9ba5-4d30-8947-c8142e07dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:26.802689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfadf8-9ba5-4d30-8947-c8142e07dd9a None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.802974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfadf8-9ba5-4d30-8947-c8142e07dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.803253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfadf8-9ba5-4d30-8947-c8142e07dd9a None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.804016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfadf8-9ba5-4d30-8947-c8142e07dd9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:26.805703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfadf8-9ba5-4d30-8947-c8142e07dd9a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:26.860791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7286d-dccd-4e2d-a9b4-7a650f5e421a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:26.870182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 126/243] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:26 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:41:26.881025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0757f-c7d9-4529-9ae9-379882e0d8c9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.881025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0757f-c7d9-4529-9ae9-379882e0d8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.881025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0757f-c7d9-4529-9ae9-379882e0d8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.895562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:26.914596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:26.915801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:26.916319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:26.916653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9dc0757f-c7d9-4529-9ae9-379882e0d8c9 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:26.917057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] PATH_INFO: `/v3/users/32ebcf90eb7d4666ac080de594611c8b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:26.917946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9dc0757f-c7d9-4529-9ae9-379882e0d8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:26.936589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] RBAC: Authorizing `identity:get_user(user_id=32ebcf90eb7d4666ac080de594611c8b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:26.939584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:26.941742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 118/244] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:26 2022] GET /identity/v3/users/32ebcf90eb7d4666ac080de594611c8b => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:26.947647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.074459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 127/245] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:27.095122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:27.112278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0757f-c7d9-4529-9ae9-379882e0d8c9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.183594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f4bb4-7529-43b2-b4a5-99eadd88762f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:27.184566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:27.185145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.185428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.185702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.248928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0757f-c7d9-4529-9ae9-379882e0d8c9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:27.264678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:27.266691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:27.289315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:27.290343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.290837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.291340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.295991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:27.306115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:27.313949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:27.315232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:27.316154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 128/246] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:27.320395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.352931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:27.379339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 119/247] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:41:27.384670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.432501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fec75-2081-4fe5-933c-c04285232a8d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:27.472056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:27.473030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.473530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.474024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.474962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:27.475486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:27.478165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:27.488357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 129/248] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:27.493833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.494914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfd5fd-839a-4bbe-96c3-3af5a17f21f6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:27.532154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:27.532672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.532957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.533234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.535218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:27.536732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:27.538505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 120/249] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:27 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:27.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.563926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:27.599097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:27.610771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9e486-8a0b-4804-96fe-fe40a431a9a2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:27.650878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:27.651604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.651897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.652178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] PATH_INFO: `/v3/users/9f474fc13a4b4397b1335234d3eb88c2` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.653384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] RBAC: Authorizing `identity:get_user(user_id=9f474fc13a4b4397b1335234d3eb88c2)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:27.654860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:27.656913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 130/250] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:27 2022] GET /identity/v3/users/9f474fc13a4b4397b1335234d3eb88c2 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:27.658416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57561a2-e819-45ad-b752-35c716779c3b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:27.664892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.697383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7ab9c-b985-479e-ac07-febc54520e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:27.698010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ab9c-b985-479e-ac07-febc54520e4a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.698292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ab9c-b985-479e-ac07-febc54520e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.698568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ab9c-b985-479e-ac07-febc54520e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.699565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7ab9c-b985-479e-ac07-febc54520e4a None admin] RBAC: Adding query filter params (role.id=85e0cedcfe2043da8a4694814f266cd5, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=32ebcf90eb7d4666ac080de594611c8b) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:27.699692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7ab9c-b985-479e-ac07-febc54520e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:27.701157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7ab9c-b985-479e-ac07-febc54520e4a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:27.708232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 121/251] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:27 2022] GET /identity/v3/role_assignments?user.id=32ebcf90eb7d4666ac080de594611c8b&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=85e0cedcfe2043da8a4694814f266cd5 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:27.741313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fc1fc-978b-4eb5-a384-8445ccee9055 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.741505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fc1fc-978b-4eb5-a384-8445ccee9055 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.741677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fc1fc-978b-4eb5-a384-8445ccee9055 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.743030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 122/252] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:27.753057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fb48-3d82-4525-92c2-c51e4787870f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.753384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fb48-3d82-4525-92c2-c51e4787870f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.753617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fb48-3d82-4525-92c2-c51e4787870f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.798846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-02b3fb48-3d82-4525-92c2-c51e4787870f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:27.800097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-02b3fb48-3d82-4525-92c2-c51e4787870f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:27.853787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb64abd-d605-43b5-802f-071b16a941b6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:27.881994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 123/253] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:27.902389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:27.904504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3fb48-3d82-4525-92c2-c51e4787870f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.919491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:27.920571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:27.921143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:27.922175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:27.923716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:27.925520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:27.931649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 131/254] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:27 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:27.933378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:27.953041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:28.021682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3fb48-3d82-4525-92c2-c51e4787870f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.070776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.071436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.071764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.072043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.079971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ba313-1f38-41c8-ab49-7489178920de None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.086569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:28.094338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:28.095283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:28.096125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:28.096739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 124/255] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:28.102265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:28.122103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b7847-a07c-432e-aa5e-1cfa894e5347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.122839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b7847-a07c-432e-aa5e-1cfa894e5347 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.123132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b7847-a07c-432e-aa5e-1cfa894e5347 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.123413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b7847-a07c-432e-aa5e-1cfa894e5347 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/9f474fc13a4b4397b1335234d3eb88c2/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.125689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b7847-a07c-432e-aa5e-1cfa894e5347 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb040a5048fb4656b0af59ad909a2338, user_id=9f474fc13a4b4397b1335234d3eb88c2, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:28.127194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b7847-a07c-432e-aa5e-1cfa894e5347 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:28.205944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:28.209805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cdc05-c085-4ea5-b0ce-4240d45e6745 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.224059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 132/256] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:27 2022] PUT /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/9f474fc13a4b4397b1335234d3eb88c2/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:28.233289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f27fd-4563-4793-b639-0b2e092e0f48 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.234475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f27fd-4563-4793-b639-0b2e092e0f48 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.234972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f27fd-4563-4793-b639-0b2e092e0f48 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.238325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 133/257] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:28.243333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509c143-87bf-49d3-bb54-2a7d4176fdec None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.243882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509c143-87bf-49d3-bb54-2a7d4176fdec None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.244363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509c143-87bf-49d3-bb54-2a7d4176fdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.248695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.249924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.250216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.250493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.251425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:28.251768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:28.253323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:28.262626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 125/258] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:28.268901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:28.269923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3509c143-87bf-49d3-bb54-2a7d4176fdec None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:28.271906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3509c143-87bf-49d3-bb54-2a7d4176fdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:28.330672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:28.359779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 134/259] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:28.383586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b4f10-e9ac-4c8c-ba99-d25ef36a9c09 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.386439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509c143-87bf-49d3-bb54-2a7d4176fdec None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:28.421478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.422510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.422798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.423077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] PATH_INFO: `/v3/users/a695db67d910450f8fabbfcec01c8327` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.425082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] RBAC: Authorizing `identity:get_user(user_id=a695db67d910450f8fabbfcec01c8327)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:28.426850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:28.429051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 126/260] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/users/a695db67d910450f8fabbfcec01c8327 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:28.435272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:28.475227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:28.518523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509c143-87bf-49d3-bb54-2a7d4176fdec None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.560110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca13f6-0e85-4159-8e80-7f0a6c6da25a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.563025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.564720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.565094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.565444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.579416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:28.585923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:28.586883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:28.587991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 135/261] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:28.595440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:28.604730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18846e-661c-4159-920f-871095747a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.605662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18846e-661c-4159-920f-871095747a0c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.606273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18846e-661c-4159-920f-871095747a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.606765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18846e-661c-4159-920f-871095747a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.656537 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:28.685892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18846e-661c-4159-920f-871095747a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:28.688775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18846e-661c-4159-920f-871095747a0c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:28.738718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e559a-cc83-48c4-9476-ea2dd8c991d3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.797817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 127/262] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:41:28.799889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:28.804684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18846e-661c-4159-920f-871095747a0c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:28.808566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.813332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.813332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.813332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.813332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:28.813332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:28.813332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:28.829052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 136/263] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:28.835185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:28.943081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4d2e6-6086-421d-b28e-2b3fc12a0067 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:28.964220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:28.978862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:28.991837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:28.991837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:28.991837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] PATH_INFO: `/v3/users/a695db67d910450f8fabbfcec01c8327` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:28.991837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] RBAC: Authorizing `identity:get_user(user_id=a695db67d910450f8fabbfcec01c8327)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:28.991837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:28.991837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 137/264] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/users/a695db67d910450f8fabbfcec01c8327 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:28.992250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:29.057734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18846e-661c-4159-920f-871095747a0c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:29.101012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b473cf4-4133-45fd-ae2c-ddf5f915f34d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:29.125392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:29.137216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:29.145014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:29.145014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:29.145014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:29.145247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:29.145247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:29.145247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:29.145247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:29.148241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:29.150011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:29.152288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 128/265] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:29.172758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:29.194652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:29.216652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:29.218893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:29.330349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 138/266] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:41:29.335637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:29.347364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692a09e-3812-417f-a255-53e68e68b91e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:29.408237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:29.408875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:29.409160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:29.409441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:29.410240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=a695db67d910450f8fabbfcec01c8327) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:29.410552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:29.412036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:29.418431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 129/267] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:29 2022] GET /identity/v3/role_assignments?user.id=a695db67d910450f8fabbfcec01c8327&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:29.462212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:29.511982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1e797-ee61-4cce-88fa-d7275e1929f7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:29.563064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4e11-380e-4638-a32e-2f7cc01f1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:29.563775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4e11-380e-4638-a32e-2f7cc01f1d10 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:29.564063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4e11-380e-4638-a32e-2f7cc01f1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:29.564343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4e11-380e-4638-a32e-2f7cc01f1d10 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:29.565619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f4e11-380e-4638-a32e-2f7cc01f1d10 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:29.567129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f4e11-380e-4638-a32e-2f7cc01f1d10 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:29.568900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 139/268] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:29 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:29.574740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:29.656383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:29.705155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a42e-c40e-4b46-851d-9f009f728383 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:29.782641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa2e36-3f9b-46dc-837c-e873bc5e6792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:29.783308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa2e36-3f9b-46dc-837c-e873bc5e6792 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:29.783616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa2e36-3f9b-46dc-837c-e873bc5e6792 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:29.783898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa2e36-3f9b-46dc-837c-e873bc5e6792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:29.784761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa2e36-3f9b-46dc-837c-e873bc5e6792 None admin] RBAC: Adding query filter params (role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=a695db67d910450f8fabbfcec01c8327) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:29.785079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa2e36-3f9b-46dc-837c-e873bc5e6792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:29.786611 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa2e36-3f9b-46dc-837c-e873bc5e6792 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:29.797233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 130/269] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:29 2022] GET /identity/v3/role_assignments?user.id=a695db67d910450f8fabbfcec01c8327&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:30.478686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a362a-34f2-438d-9526-f29ad2d151da None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.479188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a362a-34f2-438d-9526-f29ad2d151da None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.479296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a362a-34f2-438d-9526-f29ad2d151da None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.480619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 140/270] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:30.486300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994476-5c3a-47b7-98ff-f55ee0c98eb0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.486697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994476-5c3a-47b7-98ff-f55ee0c98eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.486862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994476-5c3a-47b7-98ff-f55ee0c98eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.506257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-05994476-5c3a-47b7-98ff-f55ee0c98eb0 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:30.507466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-05994476-5c3a-47b7-98ff-f55ee0c98eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:30.578244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 131/271] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:30.601092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a362a-34f2-438d-9526-f29ad2d151da None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:30.710311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a362a-34f2-438d-9526-f29ad2d151da None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:30.744788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255a1b2-c447-43d0-a04d-efb32500f0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:30.745354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a1b2-c447-43d0-a04d-efb32500f0ad None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.745633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a1b2-c447-43d0-a04d-efb32500f0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.745905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a1b2-c447-43d0-a04d-efb32500f0ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.769060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebaf40-e782-497c-bf63-a8c4effb8049 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.769354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebaf40-e782-497c-bf63-a8c4effb8049 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.769636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebaf40-e782-497c-bf63-a8c4effb8049 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.771143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 132/272] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:41:30.773113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:30.776159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eef50-3b34-4d47-8bae-ff36b15b634e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.776367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eef50-3b34-4d47-8bae-ff36b15b634e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.776652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eef50-3b34-4d47-8bae-ff36b15b634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.777239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255a1b2-c447-43d0-a04d-efb32500f0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f255a1b2-c447-43d0-a04d-efb32500f0ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:30.783014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:30.784305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:30.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:30.788257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 141/273] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:30.790157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9089164-2682-4528-b372-f4cbf1ddc65c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.790443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9089164-2682-4528-b372-f4cbf1ddc65c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.794711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9089164-2682-4528-b372-f4cbf1ddc65c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.797394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 142/274] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:30.798034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9089164-2682-4528-b372-f4cbf1ddc65c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:30.800763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-736eef50-3b34-4d47-8bae-ff36b15b634e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:30.802282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-736eef50-3b34-4d47-8bae-ff36b15b634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:30.851936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:30.870164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 133/275] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:30.872890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf26dc7-cacf-4adb-b76f-cdb16dc7177f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.873193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf26dc7-cacf-4adb-b76f-cdb16dc7177f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.873475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf26dc7-cacf-4adb-b76f-cdb16dc7177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.893582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0bf26dc7-cacf-4adb-b76f-cdb16dc7177f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:30.894769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0bf26dc7-cacf-4adb-b76f-cdb16dc7177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:30.902891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9089164-2682-4528-b372-f4cbf1ddc65c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:30.937804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2882b-2755-4952-b392-cd6a9cf6d2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:30.938420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2882b-2755-4952-b392-cd6a9cf6d2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.938700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2882b-2755-4952-b392-cd6a9cf6d2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.938975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2882b-2755-4952-b392-cd6a9cf6d2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.939717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2882b-2755-4952-b392-cd6a9cf6d2a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:30.940019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2882b-2755-4952-b392-cd6a9cf6d2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:30.941400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2882b-2755-4952-b392-cd6a9cf6d2a9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:30.949197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 143/276] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:30.953091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede589-6e6e-40c1-abdf-4e8261144c30 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:30.953383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede589-6e6e-40c1-abdf-4e8261144c30 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:30.953755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede589-6e6e-40c1-abdf-4e8261144c30 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:30.955026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 144/277] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:30.955834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede589-6e6e-40c1-abdf-4e8261144c30 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:30.962670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 134/278] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:30.963514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf26dc7-cacf-4adb-b76f-cdb16dc7177f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.023173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.063367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede589-6e6e-40c1-abdf-4e8261144c30 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.063746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf26dc7-cacf-4adb-b76f-cdb16dc7177f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.077948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.098325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.098485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f28e5-0252-4ba8-893e-f5e23d242c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.098970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.099067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f28e5-0252-4ba8-893e-f5e23d242c81 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.099257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.099352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f28e5-0252-4ba8-893e-f5e23d242c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.099556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.099663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f28e5-0252-4ba8-893e-f5e23d242c81 None admin] PATH_INFO: `/v3/users/602547d9b53443dfa3e8014dabe0fd3c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.110086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:31.113492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f28e5-0252-4ba8-893e-f5e23d242c81 None admin] RBAC: Authorizing `identity:get_user(user_id=602547d9b53443dfa3e8014dabe0fd3c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.114948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f28e5-0252-4ba8-893e-f5e23d242c81 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:31.115596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:31.116492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:31.122105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:31.122716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 135/279] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:30 2022] GET /identity/v3/users/602547d9b53443dfa3e8014dabe0fd3c => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:31.122802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 145/280] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:31.122802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.122921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58c225-233d-46f9-992b-ac7c3294cd16 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.122921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58c225-233d-46f9-992b-ac7c3294cd16 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.122921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58c225-233d-46f9-992b-ac7c3294cd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.136864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4f58c225-233d-46f9-992b-ac7c3294cd16 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:31.138058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4f58c225-233d-46f9-992b-ac7c3294cd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:31.205313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 136/281] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:31.206079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58c225-233d-46f9-992b-ac7c3294cd16 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.223776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ff218-8e99-4639-9a9f-54f97800c58d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.247506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.259186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.259861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.260146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.260418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.270477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:31.275936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:31.276893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:31.277711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 146/282] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:31.278367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.313062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58c225-233d-46f9-992b-ac7c3294cd16 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.314577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.348124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.348743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.349028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.349308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.350125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, user.id=602547d9b53443dfa3e8014dabe0fd3c, scope.system=all) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:31.350439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.351903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.356887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 137/283] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/role_assignments?user.id=602547d9b53443dfa3e8014dabe0fd3c&scope.system=all&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:31.357685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.382386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078bef8-c0ba-4ba5-9aac-7ae8bb5cfdb6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.419653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.420287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.420569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.420860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.421578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:31.421887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.423350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.432319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 147/284] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:31.433181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.478279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4d4cd-0729-4da7-a67d-72016f90dc3d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.484279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.518674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.519364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.519687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.519970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.522549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.529254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.529254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 138/285] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:31.529254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.549194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95ec6a-3d82-401e-8802-5b6da97c948a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.563322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.587515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.588248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.588487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.588728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.589448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:31.589755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.591396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.604774 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 148/286] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:31.604774 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.641102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe220-8b3b-445d-ace8-d4b4e8e6aec9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.681284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.682033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.682325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.682604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] PATH_INFO: `/v3/users/32ebcf90eb7d4666ac080de594611c8b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.683838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] RBAC: Authorizing `identity:get_user(user_id=32ebcf90eb7d4666ac080de594611c8b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.685764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.692586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 139/287] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/users/32ebcf90eb7d4666ac080de594611c8b => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:31.692586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.711737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c4751-adae-4a9b-8c84-0ee3f9551628 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.720502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.751125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.751790 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.752075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.752352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.753046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.754560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.804939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4a167-33bf-4d67-9977-00d52a5d6307 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:31.806599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:31.839354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:31.840032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:31.840325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:31.841131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] PATH_INFO: `/v3/users/9f474fc13a4b4397b1335234d3eb88c2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:31.842320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] RBAC: Authorizing `identity:get_user(user_id=9f474fc13a4b4397b1335234d3eb88c2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:31.845443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:31.847280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 140/288] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/users/9f474fc13a4b4397b1335234d3eb88c2 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:31.847883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.859695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 149/289] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:31 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:31.859695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:31.982870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d04e9b-0ac2-4cc5-a56b-f075c9afa6c7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:32.001901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2b3e-d55e-4dad-9e15-5b4fc3593ec6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:32.027281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:32.028026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:32.028313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:32.028590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:32.029974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:32.032530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:32.034381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 141/290] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:32.040351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:32.065684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:32.066639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:32.066922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:32.067196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:32.092458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:32.109464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:32.159671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:32.161178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:32.172943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9ea75-9de0-4344-b5af-2f318cc0bdeb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:32.219994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:32.220680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:32.220966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:32.221240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/32ebcf90eb7d4666ac080de594611c8b/roles/85e0cedcfe2043da8a4694814f266cd5` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:32.223363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb040a5048fb4656b0af59ad909a2338, user_id=32ebcf90eb7d4666ac080de594611c8b, role_id=85e0cedcfe2043da8a4694814f266cd5)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:32.224859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:32.276959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 150/291] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:41:32.282828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:32.293132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 142/292] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:41:32 2022] PUT /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338/users/32ebcf90eb7d4666ac080de594611c8b/roles/85e0cedcfe2043da8a4694814f266cd5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:41:32.356277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:32.409728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87089e-b7d0-45cd-aaa8-96d36404a021 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:32.460991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e63b3-bb25-4506-93cf-b909020ac2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:32.462114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e63b3-bb25-4506-93cf-b909020ac2ae None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:32.462613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e63b3-bb25-4506-93cf-b909020ac2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:32.463107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e63b3-bb25-4506-93cf-b909020ac2ae None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:32.464671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e63b3-bb25-4506-93cf-b909020ac2ae None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:32.466667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e63b3-bb25-4506-93cf-b909020ac2ae None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:32.469108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 151/293] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:32 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:32.475323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:32.595970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:32.601683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a114f-d8ea-4391-bad3-44ee2cf8a955 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:32.650382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d1cc1-6112-4061-b853-7e4f0f575bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:32.651006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d1cc1-6112-4061-b853-7e4f0f575bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:32.651292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d1cc1-6112-4061-b853-7e4f0f575bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:32.651621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d1cc1-6112-4061-b853-7e4f0f575bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:32.652469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d1cc1-6112-4061-b853-7e4f0f575bb5 None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=9f474fc13a4b4397b1335234d3eb88c2) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:32.652782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d1cc1-6112-4061-b853-7e4f0f575bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:32.654227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d1cc1-6112-4061-b853-7e4f0f575bb5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:32.670557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 143/294] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:32 2022] GET /identity/v3/role_assignments?user.id=9f474fc13a4b4397b1335234d3eb88c2&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:33.562607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5f55-42b0-4a10-8b0b-2e310f8c981a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:33.563134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5f55-42b0-4a10-8b0b-2e310f8c981a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:33.563210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5f55-42b0-4a10-8b0b-2e310f8c981a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:33.564631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 152/295] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:33.569810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066d54b-434a-4164-8e05-075ccae3ac9c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:33.570109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066d54b-434a-4164-8e05-075ccae3ac9c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:33.570397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066d54b-434a-4164-8e05-075ccae3ac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:33.588791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0066d54b-434a-4164-8e05-075ccae3ac9c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:33.590071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0066d54b-434a-4164-8e05-075ccae3ac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:33.665907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 144/296] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:33.688172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5f55-42b0-4a10-8b0b-2e310f8c981a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:33.793910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5f55-42b0-4a10-8b0b-2e310f8c981a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:33.823563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:33.829437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:33.830083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:33.830369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:33.830649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:33.841570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:33.847315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:33.848239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:33.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 153/297] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:33.854669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066d54b-434a-4164-8e05-075ccae3ac9c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:33.947795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:33.961157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066d54b-434a-4164-8e05-075ccae3ac9c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:33.997600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:33.998602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:33.999777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.000388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.001373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:34.001887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:34.003603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:34.014279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 145/298] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:34.020199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:34.123453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfaeb-2f8e-4c59-b6de-7d687b2b5e10 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:34.158827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b922ea9-538e-46d3-9765-aa8e925bce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:34.159556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922ea9-538e-46d3-9765-aa8e925bce1c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.159862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922ea9-538e-46d3-9765-aa8e925bce1c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.160194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922ea9-538e-46d3-9765-aa8e925bce1c None admin] PATH_INFO: `/v3/users/602547d9b53443dfa3e8014dabe0fd3c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.161492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b922ea9-538e-46d3-9765-aa8e925bce1c None admin] RBAC: Authorizing `identity:get_user(user_id=602547d9b53443dfa3e8014dabe0fd3c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:34.162974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b922ea9-538e-46d3-9765-aa8e925bce1c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:34.164861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 154/299] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:34 2022] GET /identity/v3/users/602547d9b53443dfa3e8014dabe0fd3c => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:34.170485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:34.302493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d67da-4265-4e66-9f94-5cd12c234320 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:34.303049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dde37-2980-49e1-8ac8-e5e9cd820d0f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.303340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dde37-2980-49e1-8ac8-e5e9cd820d0f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.303662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dde37-2980-49e1-8ac8-e5e9cd820d0f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.305316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 155/300] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:41:34.310185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c8582-090e-4301-8dc1-60fd8353b150 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.310465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c8582-090e-4301-8dc1-60fd8353b150 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.310732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c8582-090e-4301-8dc1-60fd8353b150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.331480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-164c8582-090e-4301-8dc1-60fd8353b150 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:34.332640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-164c8582-090e-4301-8dc1-60fd8353b150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:34.341918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e9893-5449-41d3-aabe-7b88738351cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:34.343449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9893-5449-41d3-aabe-7b88738351cd None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.343769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9893-5449-41d3-aabe-7b88738351cd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.344049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9893-5449-41d3-aabe-7b88738351cd None admin] PATH_INFO: `/v3/system/users/602547d9b53443dfa3e8014dabe0fd3c/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.345534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e9893-5449-41d3-aabe-7b88738351cd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=602547d9b53443dfa3e8014dabe0fd3c, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:34.346923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e9893-5449-41d3-aabe-7b88738351cd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:34.392045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:34.406320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 156/301] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:34.419411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 146/302] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 1 07:41:34 2022] PUT /identity/v3/system/users/602547d9b53443dfa3e8014dabe0fd3c/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:41:34.429158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c8582-090e-4301-8dc1-60fd8353b150 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:34.491178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241930a-ef04-4922-aa5d-5a294b61b9f0 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.491882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241930a-ef04-4922-aa5d-5a294b61b9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.492390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241930a-ef04-4922-aa5d-5a294b61b9f0 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.494041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 147/303] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:34.500999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf4771-c21a-47de-aa9c-8ed2fead0bbe None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.501293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf4771-c21a-47de-aa9c-8ed2fead0bbe None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.501576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf4771-c21a-47de-aa9c-8ed2fead0bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.521184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b4bf4771-c21a-47de-aa9c-8ed2fead0bbe None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:34.522421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b4bf4771-c21a-47de-aa9c-8ed2fead0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:34.539873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:34.557065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c8582-090e-4301-8dc1-60fd8353b150 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:34.596194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:34.596849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.597158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.597432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.610127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 148/304] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:34.614019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:34.622657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:34.623585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:34.625072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 157/305] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:34.629616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf4771-c21a-47de-aa9c-8ed2fead0bbe None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:34.632892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:34.687914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:34.742705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf4771-c21a-47de-aa9c-8ed2fead0bbe None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:34.743577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb33212-5b07-44b9-97b4-5d788f3ab729 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:34.783032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf47e-4045-4f21-b637-081c1b4f3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:34.784689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf47e-4045-4f21-b637-081c1b4f3c44 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.785187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf47e-4045-4f21-b637-081c1b4f3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.785681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf47e-4045-4f21-b637-081c1b4f3c44 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.786615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf47e-4045-4f21-b637-081c1b4f3c44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:34.787121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf47e-4045-4f21-b637-081c1b4f3c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:34.788303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c82de-4f91-4cc2-8da0-8c7a6f8503e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:34.788943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c82de-4f91-4cc2-8da0-8c7a6f8503e9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.789225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c82de-4f91-4cc2-8da0-8c7a6f8503e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.789500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c82de-4f91-4cc2-8da0-8c7a6f8503e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.789746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf47e-4045-4f21-b637-081c1b4f3c44 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:34.790636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c82de-4f91-4cc2-8da0-8c7a6f8503e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:34.792122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c82de-4f91-4cc2-8da0-8c7a6f8503e9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:34.793711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 158/306] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:34.794835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b0ea6-90d9-4512-91d7-4ce27d7c1f08 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.795118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b0ea6-90d9-4512-91d7-4ce27d7c1f08 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.795398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b0ea6-90d9-4512-91d7-4ce27d7c1f08 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.797382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 159/307] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:34.799581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b0ea6-90d9-4512-91d7-4ce27d7c1f08 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:34.803870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 149/308] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:34.805177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed691283-ae70-42d3-9b7f-4f79be6de8c1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.805469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed691283-ae70-42d3-9b7f-4f79be6de8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.805743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed691283-ae70-42d3-9b7f-4f79be6de8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.824210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ed691283-ae70-42d3-9b7f-4f79be6de8c1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:34.825402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ed691283-ae70-42d3-9b7f-4f79be6de8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:34.828780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:34.894676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 150/309] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:34.898884 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed691283-ae70-42d3-9b7f-4f79be6de8c1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:34.929079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b0ea6-90d9-4512-91d7-4ce27d7c1f08 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:34.946325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:34.966374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:34.966999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:34.967286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:34.968195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] PATH_INFO: `/v3/groups` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:34.968935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:34.970395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:35.009052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed691283-ae70-42d3-9b7f-4f79be6de8c1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:35.024427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 160/310] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 1 07:41:34 2022] POST /identity/v3/groups => generated 229 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:35.025615 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:35.065713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:35.067039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:35.067557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:35.068303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] PATH_INFO: `/v3/users/171b59d70fc74bc1b89d28d592390172` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:35.085512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:35.091889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] RBAC: Authorizing `identity:get_user(user_id=171b59d70fc74bc1b89d28d592390172)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:35.094607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:35.097483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 151/311] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:34 2022] GET /identity/v3/users/171b59d70fc74bc1b89d28d592390172 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:35.104600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:35.162002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7d981-cb5e-4d4c-8d10-267e14f1514d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:35.210602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:35.211519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:35.211869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:35.212140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:35.225970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:35.232684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:35.233729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:35.234962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 161/312] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:35.239170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:35.242522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74ee0-dfbd-4976-be33-df55614a3918 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:35.287683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:35.288302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:35.288579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:35.288849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:35.289635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] RBAC: Adding query filter params (role.id=85e0cedcfe2043da8a4694814f266cd5, user.id=171b59d70fc74bc1b89d28d592390172, scope.system=all) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:35.289944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:35.291394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:35.292546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:35.303555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 152/313] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 07:41:35 2022] GET /identity/v3/role_assignments?user.id=171b59d70fc74bc1b89d28d592390172&scope.system=all&role.id=85e0cedcfe2043da8a4694814f266cd5 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:35.376778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9312d11-8993-4b83-80f7-2103b4f65110 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:35.419421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:35.420369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:35.420896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:35.421404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:35.422307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:35.422766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:35.424372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:35.434540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 162/314] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:35.441099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:35.510101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:35.551217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1591595-432b-4c79-8a55-a4280aa7a1bd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:35.587777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:35.588619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:35.588994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:35.589424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] PATH_INFO: `/v3/users/32ebcf90eb7d4666ac080de594611c8b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:35.591095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] RBAC: Authorizing `identity:get_user(user_id=32ebcf90eb7d4666ac080de594611c8b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:35.592797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:35.594644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 153/315] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:35 2022] GET /identity/v3/users/32ebcf90eb7d4666ac080de594611c8b => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:35.600588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:35.706493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:35.712194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7a427-54ff-4ba3-aff4-8bd1eb40dc0b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:35.747278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:35.747951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:35.748241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:35.748521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:35.822016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:35.823502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:35.929279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:35.930386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 163/316] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:35.935163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:36.040890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba225b-fdbe-4367-a170-e2e66caf4206 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:36.078799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:36.084990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35d14e-e469-4474-a31f-c99a140168a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:36.085647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35d14e-e469-4474-a31f-c99a140168a1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:36.085979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35d14e-e469-4474-a31f-c99a140168a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:36.091150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35d14e-e469-4474-a31f-c99a140168a1 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:36.091150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35d14e-e469-4474-a31f-c99a140168a1 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:36.091150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35d14e-e469-4474-a31f-c99a140168a1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:36.091150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 154/317] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:35 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:36.097973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:36.209854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d490c-5e50-4b70-a52b-375c2694181c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:36.246925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33102b-b869-4451-ab1b-2003f0828e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:36.247572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33102b-b869-4451-ab1b-2003f0828e19 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:36.247867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33102b-b869-4451-ab1b-2003f0828e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:36.248149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33102b-b869-4451-ab1b-2003f0828e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:36.249061 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33102b-b869-4451-ab1b-2003f0828e19 None admin] RBAC: Adding query filter params (role.id=85e0cedcfe2043da8a4694814f266cd5, scope.project.id=eb040a5048fb4656b0af59ad909a2338, user.id=32ebcf90eb7d4666ac080de594611c8b) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:36.249378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33102b-b869-4451-ab1b-2003f0828e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:36.250891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33102b-b869-4451-ab1b-2003f0828e19 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:36.257671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 164/318] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:41:36 2022] GET /identity/v3/role_assignments?user.id=32ebcf90eb7d4666ac080de594611c8b&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=85e0cedcfe2043da8a4694814f266cd5 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:36.529153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebdca-5528-417a-a5a3-24d2851e61bb None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:36.529561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebdca-5528-417a-a5a3-24d2851e61bb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:36.529737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebdca-5528-417a-a5a3-24d2851e61bb None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:36.531172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 155/319] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:36.537032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d8667-eb04-41d6-b75f-2a4785204ff4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:36.537329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d8667-eb04-41d6-b75f-2a4785204ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:36.537608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d8667-eb04-41d6-b75f-2a4785204ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:36.556208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5c0d8667-eb04-41d6-b75f-2a4785204ff4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:36.557411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5c0d8667-eb04-41d6-b75f-2a4785204ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:36.617563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:36.628187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 165/320] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:36.650571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ebdca-5528-417a-a5a3-24d2851e61bb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:36.759229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ebdca-5528-417a-a5a3-24d2851e61bb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:36.797188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:36.797774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:36.798062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:36.798339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:36.809137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:36.814923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:36.816255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:36.817046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 156/321] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:36.826481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d8667-eb04-41d6-b75f-2a4785204ff4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:36.933017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d8667-eb04-41d6-b75f-2a4785204ff4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:36.970421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:36.971051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:36.971347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:36.971650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:36.972383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:36.972689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:36.974105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:36.983689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 166/322] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:36.989571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:37.059668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:37.106042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8125-ea19-4a10-9446-4cc6cb6cb032 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:37.145369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10f47d-edf1-46e9-b4da-d0134005b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:37.146442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10f47d-edf1-46e9-b4da-d0134005b808 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.146736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10f47d-edf1-46e9-b4da-d0134005b808 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.147018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10f47d-edf1-46e9-b4da-d0134005b808 None admin] PATH_INFO: `/v3/users/602547d9b53443dfa3e8014dabe0fd3c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.148869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10f47d-edf1-46e9-b4da-d0134005b808 None admin] RBAC: Authorizing `identity:get_user(user_id=602547d9b53443dfa3e8014dabe0fd3c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:37.151070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10f47d-edf1-46e9-b4da-d0134005b808 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:37.153910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 157/323] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:36 2022] GET /identity/v3/users/602547d9b53443dfa3e8014dabe0fd3c => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:37.162454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:37.185058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ad28b-7fc9-4d1c-a3c5-83e507533a7a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.185654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ad28b-7fc9-4d1c-a3c5-83e507533a7a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.186160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ad28b-7fc9-4d1c-a3c5-83e507533a7a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.188270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 158/324] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:37.193397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104c30-66cd-4c15-87c5-93e327bd5a5d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.193683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104c30-66cd-4c15-87c5-93e327bd5a5d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.193956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104c30-66cd-4c15-87c5-93e327bd5a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.223076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b9104c30-66cd-4c15-87c5-93e327bd5a5d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:37.224478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b9104c30-66cd-4c15-87c5-93e327bd5a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:37.247896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:37.304761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cb379-69d8-4310-a826-8f7ac4f4d20c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:37.331751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 159/325] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:37.350931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9104c30-66cd-4c15-87c5-93e327bd5a5d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:37.360136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904d270-dcc5-4040-9168-550485caff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:37.361221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904d270-dcc5-4040-9168-550485caff22 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.361722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904d270-dcc5-4040-9168-550485caff22 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.362011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904d270-dcc5-4040-9168-550485caff22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.362822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904d270-dcc5-4040-9168-550485caff22 None admin] RBAC: Adding query filter params (role.id=338a254b1f394ba2b6d9f5fb3885cc4d, user.id=602547d9b53443dfa3e8014dabe0fd3c, scope.system=all) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:37.363137 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904d270-dcc5-4040-9168-550485caff22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:37.365039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904d270-dcc5-4040-9168-550485caff22 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:37.382000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 167/326] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 07:41:37 2022] GET /identity/v3/role_assignments?user.id=602547d9b53443dfa3e8014dabe0fd3c&scope.system=all&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:37.434127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:37.465403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975544-7292-4576-9346-dbc0da6847d0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.465716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975544-7292-4576-9346-dbc0da6847d0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.465983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975544-7292-4576-9346-dbc0da6847d0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.467718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 168/327] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:37.473558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecc5b0-3f78-4802-8b4a-5b35d6e56704 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.473851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecc5b0-3f78-4802-8b4a-5b35d6e56704 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.474159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecc5b0-3f78-4802-8b4a-5b35d6e56704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.497529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-20ecc5b0-3f78-4802-8b4a-5b35d6e56704 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:37.499462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9104c30-66cd-4c15-87c5-93e327bd5a5d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:37.499828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-20ecc5b0-3f78-4802-8b4a-5b35d6e56704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:37.551760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:37.553039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.553330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.553697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.555935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:37.557669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:37.559588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 160/328] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:37.566065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:37.598007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 169/329] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:37.622803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecc5b0-3f78-4802-8b4a-5b35d6e56704 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:37.671249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:37.706881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28d3d0-9c84-4aab-9cdb-0adcc22edd77 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:37.732752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:37.738529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecc5b0-3f78-4802-8b4a-5b35d6e56704 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:37.745176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:37.745808 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.746201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.747008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] PATH_INFO: `/v3/groups` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.748230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:37.750170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:37.780555 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:37.781238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.781565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.781851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.792786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:37.798132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:37.799408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:37.800415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 170/330] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:37.805294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:37.847673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 161/331] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 1 07:41:37 2022] POST /identity/v3/groups => generated 226 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:41:37.912000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7630f-3c9c-4aaf-aea9-299008e80177 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:37.913720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:37.948981 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:37.949624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:37.949909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:37.950201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:37.950920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:37.951228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:37.953290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:37.962388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 171/332] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:37.968576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:38.073640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24370c0-54d6-4ba9-b17c-cdf19d56ea87 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:38.109654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcaaf2b-1394-4567-a8c0-182e7edeedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:38.110304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaaf2b-1394-4567-a8c0-182e7edeedfa None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:38.110584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaaf2b-1394-4567-a8c0-182e7edeedfa None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:38.110853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaaf2b-1394-4567-a8c0-182e7edeedfa None admin] PATH_INFO: `/v3/users/171b59d70fc74bc1b89d28d592390172` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:38.112135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcaaf2b-1394-4567-a8c0-182e7edeedfa None admin] RBAC: Authorizing `identity:get_user(user_id=171b59d70fc74bc1b89d28d592390172)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:38.113594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcaaf2b-1394-4567-a8c0-182e7edeedfa None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:38.115445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 162/333] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:37 2022] GET /identity/v3/users/171b59d70fc74bc1b89d28d592390172 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:38.120887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:38.237553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14026a7d-dbce-424b-a274-465e90c5b54f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:38.273904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956904-84c5-4814-932e-745e02de9c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:38.274589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956904-84c5-4814-932e-745e02de9c4e None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:38.274980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956904-84c5-4814-932e-745e02de9c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:38.275271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956904-84c5-4814-932e-745e02de9c4e None admin] PATH_INFO: `/v3/system/users/171b59d70fc74bc1b89d28d592390172/roles/85e0cedcfe2043da8a4694814f266cd5` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:38.276796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956904-84c5-4814-932e-745e02de9c4e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=171b59d70fc74bc1b89d28d592390172, role_id=85e0cedcfe2043da8a4694814f266cd5)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:38.278245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956904-84c5-4814-932e-745e02de9c4e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:38.322191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 172/334] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 1 07:41:38 2022] PUT /identity/v3/system/users/171b59d70fc74bc1b89d28d592390172/roles/85e0cedcfe2043da8a4694814f266cd5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:40.303291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9f5d-cc9d-45c6-a4a0-34ed31d3d163 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:40.303843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9f5d-cc9d-45c6-a4a0-34ed31d3d163 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:40.304241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9f5d-cc9d-45c6-a4a0-34ed31d3d163 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:40.306070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 163/335] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:40.311069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd492f-8140-4e51-93fe-5b5bdfeae30b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:40.311996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd492f-8140-4e51-93fe-5b5bdfeae30b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:40.312175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd492f-8140-4e51-93fe-5b5bdfeae30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:40.335304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9dcd492f-8140-4e51-93fe-5b5bdfeae30b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:40.336570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9dcd492f-8140-4e51-93fe-5b5bdfeae30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:40.371353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:40.429869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 173/336] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:40.463819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e9f5d-cc9d-45c6-a4a0-34ed31d3d163 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:40.529405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:40.591987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e9f5d-cc9d-45c6-a4a0-34ed31d3d163 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:40.594603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683cfb1-e5e7-43a0-80e5-e8c935d625a3 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:40.595272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683cfb1-e5e7-43a0-80e5-e8c935d625a3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:40.595903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683cfb1-e5e7-43a0-80e5-e8c935d625a3 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:40.599885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 174/337] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:40.606410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd48e9a-d5df-45a7-8c7d-39481e14004c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:40.607008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd48e9a-d5df-45a7-8c7d-39481e14004c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:40.607487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd48e9a-d5df-45a7-8c7d-39481e14004c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:40.633428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bfd48e9a-d5df-45a7-8c7d-39481e14004c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:40.634976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bfd48e9a-d5df-45a7-8c7d-39481e14004c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:40.639925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:40.640696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:40.640989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:40.641270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:40.657802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:40.666522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:40.667420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:40.668890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 164/338] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:40.673961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:40.751785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 175/339] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:40.836394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3a0cc-9aad-430f-9697-ad9bc2ef0148 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:40.854952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd48e9a-d5df-45a7-8c7d-39481e14004c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:40.857579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:40.893926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:40.894931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:40.895417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:40.896083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:40.898110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:40.898651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:40.900320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:40.913765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 165/340] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:40.925176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:41.052042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:41.055624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c48a5-c3f8-4b42-84a7-297d4cefe347 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:41.071698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd48e9a-d5df-45a7-8c7d-39481e14004c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:41.120348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23416304-1eca-4665-8024-ad996cdd41e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:41.121384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23416304-1eca-4665-8024-ad996cdd41e8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.121874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23416304-1eca-4665-8024-ad996cdd41e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.122349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23416304-1eca-4665-8024-ad996cdd41e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.148281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:41.148961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.149254 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.149538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.176562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:41.180076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:41.185070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:41.186707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:41.187734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 176/341] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:41.195719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:41.221937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23416304-1eca-4665-8024-ad996cdd41e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:41.224665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23416304-1eca-4665-8024-ad996cdd41e8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:41.365358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d847d26-db7e-40b1-acc6-14bb59d2b99d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:41.377012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 166/342] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:41:41.379104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811a5a-e6f0-4105-ae60-5320d07cdfdb None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.379639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811a5a-e6f0-4105-ae60-5320d07cdfdb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.380152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811a5a-e6f0-4105-ae60-5320d07cdfdb None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.385832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 167/343] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:41.385832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13ece-bd30-49c1-989f-a1c99a02cc24 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.385832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13ece-bd30-49c1-989f-a1c99a02cc24 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.385832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13ece-bd30-49c1-989f-a1c99a02cc24 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.386827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 168/344] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:41.387900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13ece-bd30-49c1-989f-a1c99a02cc24 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:41.404576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227c241-2f12-44c1-86a0-dbc7fc40b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:41.405769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227c241-2f12-44c1-86a0-dbc7fc40b952 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.406052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227c241-2f12-44c1-86a0-dbc7fc40b952 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.406420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227c241-2f12-44c1-86a0-dbc7fc40b952 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.407105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227c241-2f12-44c1-86a0-dbc7fc40b952 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:41.407412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227c241-2f12-44c1-86a0-dbc7fc40b952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:41.409565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227c241-2f12-44c1-86a0-dbc7fc40b952 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:41.423557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 177/345] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:41.423557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201a97-f326-4de3-875e-f95e5b84d018 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.423557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201a97-f326-4de3-875e-f95e5b84d018 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.423557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201a97-f326-4de3-875e-f95e5b84d018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.442753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-84201a97-f326-4de3-875e-f95e5b84d018 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:41.444002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-84201a97-f326-4de3-875e-f95e5b84d018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:41.447056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:41.483679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:41.511856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13ece-bd30-49c1-989f-a1c99a02cc24 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:41.518786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 178/346] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:41.520443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790599b2-8c9b-4005-8f33-453181f92614 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.520734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790599b2-8c9b-4005-8f33-453181f92614 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.523889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790599b2-8c9b-4005-8f33-453181f92614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.540459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-790599b2-8c9b-4005-8f33-453181f92614 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:41.544561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-790599b2-8c9b-4005-8f33-453181f92614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:41.559948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:41.561021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.561886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.562573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.564821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:41.566551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:41.568500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 169/347] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:41 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:41.569550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:41.618641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 179/348] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:41.619505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790599b2-8c9b-4005-8f33-453181f92614 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:41.680692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef8510-aabe-47f5-903a-687bcd95e6c0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:41.705745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:41.719426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:41.720481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.728588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.728588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.734707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:41.741909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790599b2-8c9b-4005-8f33-453181f92614 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:41.796982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:41.797810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.798167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.798515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:41.800810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:41.802485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:41.813553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:41.818662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:41.819726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 180/349] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:41.822990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:41.943867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 170/350] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:41.945172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:41.950048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d70ec-bda2-4262-88f1-6b9a8c578ee2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:41.989400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:41.994729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2beff-ea21-4633-88dc-d685c05d8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:41.995489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2beff-ea21-4633-88dc-d685c05d8800 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:41.995830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2beff-ea21-4633-88dc-d685c05d8800 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:41.996112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2beff-ea21-4633-88dc-d685c05d8800 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.003788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2beff-ea21-4633-88dc-d685c05d8800 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:42.005848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2beff-ea21-4633-88dc-d685c05d8800 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:42.007406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 181/351] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:41 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:42.009222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6697-2a4c-481d-9017-12bfa96471fc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.009516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6697-2a4c-481d-9017-12bfa96471fc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.009800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6697-2a4c-481d-9017-12bfa96471fc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.011806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 182/352] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:42.012831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c6697-2a4c-481d-9017-12bfa96471fc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.012941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:42.082233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff19bb-9200-480f-b654-e3f2d265e36a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.123061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.123773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.124060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.124334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.136733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:42.142529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:42.143702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 171/353] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:42.144818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.153058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c6697-2a4c-481d-9017-12bfa96471fc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.201573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.202124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.202407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.202689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.203435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:42.203771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:42.205157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:42.218552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 183/354] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:42.218552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.283872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd62b6-1d99-4993-b5dd-d6ef77469501 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.285397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:42.285534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:42.327568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6eb446-bd05-4e28-a26f-ce76db7c5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.328249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eb446-bd05-4e28-a26f-ce76db7c5162 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.328530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eb446-bd05-4e28-a26f-ce76db7c5162 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.328804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eb446-bd05-4e28-a26f-ce76db7c5162 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.330004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eb446-bd05-4e28-a26f-ce76db7c5162 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:42.331655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eb446-bd05-4e28-a26f-ce76db7c5162 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:42.333550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 172/355] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:42.334458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b7494-0fc1-4560-9e26-3c19afc8795d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.334748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b7494-0fc1-4560-9e26-3c19afc8795d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.335029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b7494-0fc1-4560-9e26-3c19afc8795d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.341024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9c50a-9e5a-413d-b6d8-c0799918afc0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.356451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-216b7494-0fc1-4560-9e26-3c19afc8795d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:42.357698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-216b7494-0fc1-4560-9e26-3c19afc8795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:42.383419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.384080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.384371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.384654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.385460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:42.385776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:42.387234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:42.393739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 184/356] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:42.394548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.447216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 173/357] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:42.448207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b7494-0fc1-4560-9e26-3c19afc8795d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.530582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71e594-5f60-47d2-a4ca-35c229ee500a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.558467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:42.572861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:42.573283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96d30f-b7ad-4411-82f4-2b41efaed7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.574407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96d30f-b7ad-4411-82f4-2b41efaed7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.574859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96d30f-b7ad-4411-82f4-2b41efaed7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.579085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96d30f-b7ad-4411-82f4-2b41efaed7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.579085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96d30f-b7ad-4411-82f4-2b41efaed7c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:42.579461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b7494-0fc1-4560-9e26-3c19afc8795d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.579568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96d30f-b7ad-4411-82f4-2b41efaed7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:42.579568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96d30f-b7ad-4411-82f4-2b41efaed7c4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:42.593392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 185/358] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:42.595488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8d86-3249-491c-84bf-dde29a21ffdc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.596174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8d86-3249-491c-84bf-dde29a21ffdc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.596635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8d86-3249-491c-84bf-dde29a21ffdc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.598285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 186/359] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:42.599048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c8d86-3249-491c-84bf-dde29a21ffdc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.624502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.625299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.625805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.626262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] PATH_INFO: `/v3/users/171b59d70fc74bc1b89d28d592390172` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.627669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] RBAC: Authorizing `identity:get_user(user_id=171b59d70fc74bc1b89d28d592390172)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:42.629507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:42.635614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 174/360] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/users/171b59d70fc74bc1b89d28d592390172 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:42.635614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.738307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c8d86-3249-491c-84bf-dde29a21ffdc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.780545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540d44-71ff-474b-9acd-92ade75aa8df None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.791684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.792275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.792860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.792860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.793994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:42.795497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:42.798605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 187/361] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:42.799461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.829813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d606b3-9943-48fe-b292-780277569cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.830474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d606b3-9943-48fe-b292-780277569cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.831003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d606b3-9943-48fe-b292-780277569cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.831003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d606b3-9943-48fe-b292-780277569cf3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.847184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d606b3-9943-48fe-b292-780277569cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-91d606b3-9943-48fe-b292-780277569cf3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:42.852330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:42.853228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:42.854552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 175/362] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:42.855427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729683e-b1b0-4f61-9098-dd1070bb6e72 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.855427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729683e-b1b0-4f61-9098-dd1070bb6e72 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.855427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729683e-b1b0-4f61-9098-dd1070bb6e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:42.866486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:42.877177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7729683e-b1b0-4f61-9098-dd1070bb6e72 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:42.878388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7729683e-b1b0-4f61-9098-dd1070bb6e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:42.910609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:42.934320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9048d85-8df2-4666-86a3-3202bf1170ed None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:42.962832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 176/363] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:42.963723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729683e-b1b0-4f61-9098-dd1070bb6e72 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:42.985091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:42.985984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:42.986277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:42.986965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.071259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:43.072967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:43.093874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729683e-b1b0-4f61-9098-dd1070bb6e72 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:43.141153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:43.141848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:43.141977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:43.142143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:43.142406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.143214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] RBAC: Adding query filter params (role.id=85e0cedcfe2043da8a4694814f266cd5, user.id=171b59d70fc74bc1b89d28d592390172, scope.system=all) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:43.143547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:43.145076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:43.160858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 177/364] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/role_assignments?user.id=171b59d70fc74bc1b89d28d592390172&scope.system=all&role.id=85e0cedcfe2043da8a4694814f266cd5 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:43.161810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:43.193517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:43.218874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 188/365] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:43.219775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:43.300694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576480c-b646-4388-8996-1c7101acd67d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:43.344976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e600f-36d1-4c66-a563-bb4e6bf457c1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:43.346350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:43.346998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:43.347286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:43.347920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.348759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:43.350091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:43.351730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:43.360113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 178/366] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:43 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:43.360475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:43.409936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:43.411430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:43.412087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:43.412438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.413272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:43.414890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:43.416505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:43.433893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 189/367] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:43.435117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:43.466718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:43.472932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:43.499822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8080895-2ca5-4adb-b751-54ff53544471 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:43.552793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:43.553472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:43.553751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:43.554026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.569593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:43.577044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:43.578179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:43.579117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:43.581780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 179/368] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:43.585706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:43.651285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7f29a-13a2-4ac4-ab2d-d3656203c592 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:43.700071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:43.701037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:43.701892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:43.702948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.704382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:43.706035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:43.708733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 190/369] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:43 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:43.711491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:43.730873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f346c11-9d3b-4d1e-90a0-a78798807aa9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:43.773729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:43.774371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:43.774657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:43.774933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.789647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:43.815424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:43.838519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddea183-c0c0-44e1-8290-f14a2016992d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:43.865107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:43.866601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:43.882856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:43.883486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:43.883806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:43.884087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:43.884814 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:43.885121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:43.886527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:43.896112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 191/370] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:43.897019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.006720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 180/371] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:44.007518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.030093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e596284-8578-4607-8c3f-acc6df9fd30f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.075514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:44.093149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.093149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.093149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.093704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.094819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.100337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.100337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 192/372] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:43 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:44.100337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.113700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:44.147657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e156cd-b022-4824-9386-06dab37ec055 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.185180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.186559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.186914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.187396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.218594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648a89c-c540-41ac-a592-15582abde17a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.262609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.262886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.263734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.264689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.265107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.265237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.267594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.269317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.271172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 193/373] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:44.272126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.307414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:44.374262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 181/374] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:44.375560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.388458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:44.412390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2f59d-4860-4f17-a0e3-0686e312af9b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.450630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.452186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.452667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.453135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.454285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=eb040a5048fb4656b0af59ad909a2338) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:44.454895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.456579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.462850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 194/375] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:44.463964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.489969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a126b5d-adce-433a-a37a-1800f2157cb7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.528738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.529643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.529989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.530212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.531490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.533227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.536406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 182/376] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:44.543939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.561382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:44.585030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d741140-4c7b-4071-8d4f-3d3658e4ac9b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.627524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.628930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.629429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.629902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] PATH_INFO: `/v3/projects/5b4d36fd27fd480aad590e2c1f973948` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.631574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] RBAC: Authorizing `identity:get_project(project_id=5b4d36fd27fd480aad590e2c1f973948)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.634035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.636786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 195/377] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/projects/5b4d36fd27fd480aad590e2c1f973948 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:44.643625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.661951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:44.675265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba9e40-02f0-41d3-8093-a69a89eb5620 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.719409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0ff5-a7a3-4a72-9fe0-5fad8b844c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.720083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0ff5-a7a3-4a72-9fe0-5fad8b844c90 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.720429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0ff5-a7a3-4a72-9fe0-5fad8b844c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.720706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0ff5-a7a3-4a72-9fe0-5fad8b844c90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.721730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0ff5-a7a3-4a72-9fe0-5fad8b844c90 None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=eb040a5048fb4656b0af59ad909a2338) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:44.722052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0ff5-a7a3-4a72-9fe0-5fad8b844c90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.725100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0ff5-a7a3-4a72-9fe0-5fad8b844c90 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.732940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 183/378] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:44.734534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ade52-5c2e-4027-8004-7c686a185c00 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.734830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ade52-5c2e-4027-8004-7c686a185c00 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.735108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ade52-5c2e-4027-8004-7c686a185c00 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.736625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 184/379] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:44.743614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952530b-dc33-4275-86f1-dddb025fa056 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.743614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952530b-dc33-4275-86f1-dddb025fa056 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.743614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952530b-dc33-4275-86f1-dddb025fa056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.786191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c952530b-dc33-4275-86f1-dddb025fa056 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:44.787374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c952530b-dc33-4275-86f1-dddb025fa056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:44.844567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e990f4-781f-44a0-b4e3-22ab2eb630bf None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:44.899313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 185/380] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:44.921256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:44.921920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:44.922210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:44.922491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] PATH_INFO: `/v3/groups/51baed48ed134368906d54b9ce9b6e23` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:44.925475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952530b-dc33-4275-86f1-dddb025fa056 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:44.928451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:44.929833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] RBAC: Authorizing `identity:get_group(group_id=51baed48ed134368906d54b9ce9b6e23)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:44.931346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:44.939793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 196/381] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/groups/51baed48ed134368906d54b9ce9b6e23 => generated 229 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:44.941504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:45.052593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:45.062753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952530b-dc33-4275-86f1-dddb025fa056 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:45.080594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1b09-8123-4b85-88a9-ff6078d4a29d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:45.100380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:45.101311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:45.101805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:45.102239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:45.113728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:45.119523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:45.120541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:45.121333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:45.124434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 186/382] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:45.125398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:45.126231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:45.126673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:45.127107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:45.128094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] RBAC: Adding query filter params (group.id=51baed48ed134368906d54b9ce9b6e23, role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=5b4d36fd27fd480aad590e2c1f973948) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:45.128584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:45.129865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:45.132194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:45.148827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 197/383] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:44 2022] GET /identity/v3/role_assignments?group.id=51baed48ed134368906d54b9ce9b6e23&scope.project.id=5b4d36fd27fd480aad590e2c1f973948&role.id=05421dac98a848f68d13ca21036dbfbb => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:45.206667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:45.250816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc26e2f-9cb1-4a49-8a6d-0060f0d4f17b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:45.293423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:45.294065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:45.294348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:45.294621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:45.295660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:45.295971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:45.297555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:45.309117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 187/384] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:45.315222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:45.433797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37633e4-19a8-489f-b2f6-e0fcbb5c683d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:45.470955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:45.471639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:45.471928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:45.472209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:45.473474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:45.474906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:45.476842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 198/385] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:45 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:45.482314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:45.612130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fb4b1-a3c9-4f1e-9458-4b64474ab08f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:45.637164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:45.650191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000acf73-8af7-466d-8276-81b57fc15ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:45.650847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000acf73-8af7-466d-8276-81b57fc15ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:45.651132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000acf73-8af7-466d-8276-81b57fc15ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:45.651410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000acf73-8af7-466d-8276-81b57fc15ad5 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:45.652770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000acf73-8af7-466d-8276-81b57fc15ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:45.667967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000acf73-8af7-466d-8276-81b57fc15ad5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:45.667967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 188/386] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:45 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:45.668156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:45.677007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:45.796846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59024e19-1a0e-4d66-b4c6-01f6312c3903 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:45.831292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e3f6-43b6-4ea1-b874-dca0f214a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:45.832005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e3f6-43b6-4ea1-b874-dca0f214a308 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:45.832292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e3f6-43b6-4ea1-b874-dca0f214a308 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:45.832571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e3f6-43b6-4ea1-b874-dca0f214a308 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/groups/89b7c81b88df4e7791116c183eccc372/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:45.834548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e3f6-43b6-4ea1-b874-dca0f214a308 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b, group_id=89b7c81b88df4e7791116c183eccc372, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:45.836041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e3f6-43b6-4ea1-b874-dca0f214a308 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:45.938726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 199/387] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 07:41:45 2022] PUT /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/groups/89b7c81b88df4e7791116c183eccc372/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:46.203195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fd5bb-eb4d-45d0-a28f-5dad1f5aab7a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:46.203579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fd5bb-eb4d-45d0-a28f-5dad1f5aab7a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:46.203756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fd5bb-eb4d-45d0-a28f-5dad1f5aab7a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:46.205354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 189/388] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:46.210546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd1af4-0c5b-43c2-b636-fc07201777f4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:46.210842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd1af4-0c5b-43c2-b636-fc07201777f4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:46.211127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd1af4-0c5b-43c2-b636-fc07201777f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:46.239385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-92bd1af4-0c5b-43c2-b636-fc07201777f4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:46.240628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92bd1af4-0c5b-43c2-b636-fc07201777f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:46.315408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:46.327294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 200/389] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:46.362806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fd5bb-eb4d-45d0-a28f-5dad1f5aab7a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:46.525256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fd5bb-eb4d-45d0-a28f-5dad1f5aab7a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:46.560051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:46.560936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:46.561284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:46.561568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:46.572372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:46.578112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:46.578998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:46.579836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 190/390] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:46.588640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd1af4-0c5b-43c2-b636-fc07201777f4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:46.727314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd1af4-0c5b-43c2-b636-fc07201777f4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:46.785709 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:46.786237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:46.786522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:46.786793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:46.787509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:46.787844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:46.789262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:46.800886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 201/391] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:46.806337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:46.893107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:46.961404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c823-c225-433a-8426-2bbbbf091f0c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:47.003992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132b9d-3138-4d29-b37e-1a7be9d9ead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:47.004639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132b9d-3138-4d29-b37e-1a7be9d9ead4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.004920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132b9d-3138-4d29-b37e-1a7be9d9ead4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.005196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132b9d-3138-4d29-b37e-1a7be9d9ead4 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.006354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132b9d-3138-4d29-b37e-1a7be9d9ead4 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:47.007845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132b9d-3138-4d29-b37e-1a7be9d9ead4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:47.009760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 191/392] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:46 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:47.015918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:47.113481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:47.174482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214ad80-2775-4af1-a0df-66f9525dfc96 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:47.184688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7265b3-5717-47cf-b5b9-b1fbc2dc92f6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.184980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7265b3-5717-47cf-b5b9-b1fbc2dc92f6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.185244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7265b3-5717-47cf-b5b9-b1fbc2dc92f6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.187392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 192/393] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:47.199478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e30fde-447f-401d-9918-64f5bfff3df6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.199805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e30fde-447f-401d-9918-64f5bfff3df6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.200087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e30fde-447f-401d-9918-64f5bfff3df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.226693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:47.227350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.227660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.227941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.230979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:47.231657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:47.233835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 202/394] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:47 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:47.237727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-87e30fde-447f-401d-9918-64f5bfff3df6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:47.239029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-87e30fde-447f-401d-9918-64f5bfff3df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:47.240413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:47.371409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 193/395] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:47.380831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c501c-6b2a-4af7-be9b-98af7efc491d None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.381118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c501c-6b2a-4af7-be9b-98af7efc491d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.381399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c501c-6b2a-4af7-be9b-98af7efc491d None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.383005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 194/396] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:47.400897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5426b8-b5ac-47ef-b24f-d5fe1b82af41 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.401181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5426b8-b5ac-47ef-b24f-d5fe1b82af41 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.401457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5426b8-b5ac-47ef-b24f-d5fe1b82af41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.404924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048eef-35f2-49d9-a6cc-11f9a084db62 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:47.422684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0a5426b8-b5ac-47ef-b24f-d5fe1b82af41 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:47.423907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0a5426b8-b5ac-47ef-b24f-d5fe1b82af41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:47.437407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:47.448984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:47.450087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.450701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.451024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/groups/89b7c81b88df4e7791116c183eccc372/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.455105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b, group_id=89b7c81b88df4e7791116c183eccc372, role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:47.457664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:47.475300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:47.502275 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 195/397] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:47.503389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5426b8-b5ac-47ef-b24f-d5fe1b82af41 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:47.524604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 203/398] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 07:41:47 2022] PUT /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b/groups/89b7c81b88df4e7791116c183eccc372/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:47.531216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:47.658299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0eb0f-4479-41ce-b98b-a3a5c146854c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:47.707716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412c383-62e4-466b-8c2c-1fcf55717d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:47.708641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412c383-62e4-466b-8c2c-1fcf55717d6b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.708931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412c383-62e4-466b-8c2c-1fcf55717d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.709647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412c383-62e4-466b-8c2c-1fcf55717d6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.712199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5426b8-b5ac-47ef-b24f-d5fe1b82af41 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:47.728531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412c383-62e4-466b-8c2c-1fcf55717d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e412c383-62e4-466b-8c2c-1fcf55717d6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:47.736241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:47.737173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:47.737985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:47.740921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 204/399] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:47.740921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c572ad-aac7-483e-9f7a-96ea8dfbae90 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.741178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c572ad-aac7-483e-9f7a-96ea8dfbae90 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.746794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c572ad-aac7-483e-9f7a-96ea8dfbae90 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.746794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 205/400] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:47.747988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c572ad-aac7-483e-9f7a-96ea8dfbae90 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:47.771079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd15aae-0481-48df-903e-07fdfb89a25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:47.771821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd15aae-0481-48df-903e-07fdfb89a25d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.772096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd15aae-0481-48df-903e-07fdfb89a25d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.772379 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd15aae-0481-48df-903e-07fdfb89a25d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.784633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd15aae-0481-48df-903e-07fdfb89a25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8dd15aae-0481-48df-903e-07fdfb89a25d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:47.789667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:47.790574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:47.791395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 196/401] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:47.792533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136910-10ce-4ed5-9f82-13b97a2996a6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.792533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136910-10ce-4ed5-9f82-13b97a2996a6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.792728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136910-10ce-4ed5-9f82-13b97a2996a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.812994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:47.817717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-47136910-10ce-4ed5-9f82-13b97a2996a6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:47.818937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-47136910-10ce-4ed5-9f82-13b97a2996a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:47.872451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c572ad-aac7-483e-9f7a-96ea8dfbae90 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:47.909915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 197/402] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:47.910765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47136910-10ce-4ed5-9f82-13b97a2996a6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:47.912285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:47.913202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:47.913813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:47.913917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:47.914401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:47.915484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:47.916236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:47.917998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:47.938311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 206/403] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:47.944041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.051604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47136910-10ce-4ed5-9f82-13b97a2996a6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.051796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d20411-8c1c-4c63-bef6-5dd176584a0f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.063283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:48.094713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.095371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.095713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.095997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.096267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.097200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.097618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.098137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.098423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.098949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.099157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:48.099467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.100766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 207/404] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:47 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:48.100867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.101369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.117193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 198/405] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:48.118184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.151593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:48.231626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0af876-1e05-4e1e-9895-8fee91d3c997 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.239978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35d20a-26ed-461d-93c5-7985d1272a31 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.276998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.277839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.278177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.278453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.278979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.279697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.279983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.280305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.282172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.283813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.285658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 199/406] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:48 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:48.286440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.292200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:48.299675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:48.300581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:48.301369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 208/407] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:48.302567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.339054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:48.406589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:48.407275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b952-c5a4-43c3-8c00-889dcf3f192f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.410077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec4deb-adb3-412a-aa1a-66c6ae25d851 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.443271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752901a-d2d4-4853-8465-448682face50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.443980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752901a-d2d4-4853-8465-448682face50 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.444265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752901a-d2d4-4853-8465-448682face50 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.444640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752901a-d2d4-4853-8465-448682face50 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338/groups/89b7c81b88df4e7791116c183eccc372/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.447022 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752901a-d2d4-4853-8465-448682face50 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb040a5048fb4656b0af59ad909a2338, group_id=89b7c81b88df4e7791116c183eccc372, role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.447891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.448500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752901a-d2d4-4853-8465-448682face50 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.448651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.449011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.449356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.450706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.453658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.459573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 200/408] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:48 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:48.459573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.552469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 209/409] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 07:41:48 2022] PUT /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338/groups/89b7c81b88df4e7791116c183eccc372/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:48.553547 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752901a-d2d4-4853-8465-448682face50 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.573867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf79de5-7b5a-4d54-b7a3-69c01009ad79 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.624415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36c7c6-c2fb-42f9-9a7a-f607c93976f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.625974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36c7c6-c2fb-42f9-9a7a-f607c93976f1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.626489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36c7c6-c2fb-42f9-9a7a-f607c93976f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.627325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36c7c6-c2fb-42f9-9a7a-f607c93976f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.628654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36c7c6-c2fb-42f9-9a7a-f607c93976f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:48.629181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36c7c6-c2fb-42f9-9a7a-f607c93976f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.632080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36c7c6-c2fb-42f9-9a7a-f607c93976f1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.645617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 201/410] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:48.647782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246181-4c38-49b2-a753-aa3121221c4f None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.648256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246181-4c38-49b2-a753-aa3121221c4f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.649198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246181-4c38-49b2-a753-aa3121221c4f None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.656238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 202/411] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:48.658157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246181-4c38-49b2-a753-aa3121221c4f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.696745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:48.699890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:48.706626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752901a-d2d4-4853-8465-448682face50 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.764480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef47e61-f876-4636-89bb-5e47a7a77211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.765160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef47e61-f876-4636-89bb-5e47a7a77211 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.765446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef47e61-f876-4636-89bb-5e47a7a77211 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.765727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef47e61-f876-4636-89bb-5e47a7a77211 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.768058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef47e61-f876-4636-89bb-5e47a7a77211 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.769561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef47e61-f876-4636-89bb-5e47a7a77211 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.776051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 210/412] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:48 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:48.776051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5ecae-f981-4b7b-ab16-9b429d2c07b4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.776051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5ecae-f981-4b7b-ab16-9b429d2c07b4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.776051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5ecae-f981-4b7b-ab16-9b429d2c07b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.793093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-dfb5ecae-f981-4b7b-ab16-9b429d2c07b4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:48.794448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dfb5ecae-f981-4b7b-ab16-9b429d2c07b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:48.801590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246181-4c38-49b2-a753-aa3121221c4f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:48.846237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:48.846848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:48.847294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:48.847719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] PATH_INFO: `/v3/groups/51baed48ed134368906d54b9ce9b6e23` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:48.848965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] RBAC: Authorizing `identity:get_group(group_id=51baed48ed134368906d54b9ce9b6e23)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:48.850618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:48.856910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 203/413] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:48 2022] GET /identity/v3/groups/51baed48ed134368906d54b9ce9b6e23 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:48.857904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.874805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 211/414] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:48.883747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5ecae-f981-4b7b-ab16-9b429d2c07b4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:48.993957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:48.997538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f63b-6801-4b54-91d0-da8b9a114eb8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:49.010653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:49.017992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5ecae-f981-4b7b-ab16-9b429d2c07b4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:49.033287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:49.034025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.034711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.035193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338/groups/89b7c81b88df4e7791116c183eccc372/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.037843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb040a5048fb4656b0af59ad909a2338, group_id=89b7c81b88df4e7791116c183eccc372, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:49.039301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:49.057459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:49.058374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.058839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.059383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] PATH_INFO: `/v3/projects/5b4d36fd27fd480aad590e2c1f973948` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.061630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] RBAC: Authorizing `identity:get_project(project_id=5b4d36fd27fd480aad590e2c1f973948)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:49.063311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:49.065400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 212/415] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:48 2022] GET /identity/v3/projects/5b4d36fd27fd480aad590e2c1f973948 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:49.066305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:49.144233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 204/416] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 07:41:48 2022] PUT /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338/groups/89b7c81b88df4e7791116c183eccc372/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:49.145453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:49.204583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bab1f-cbf2-434b-bc05-8dfeb878c36f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:49.283621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:49.284297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.284582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.284864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.306924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:49.320343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:49.321361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:49.322205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 213/417] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:49.341958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8934c0-ec1a-464e-b0d8-db088f56d219 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:49.345219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:49.375457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:49.383061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:49.384112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.384601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.385175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] PATH_INFO: `/v3/projects/5b4d36fd27fd480aad590e2c1f973948/groups/51baed48ed134368906d54b9ce9b6e23/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.387325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b4d36fd27fd480aad590e2c1f973948, group_id=51baed48ed134368906d54b9ce9b6e23, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:49.389662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:49.467515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa16151-c93b-4a03-b18b-1958b0c59bf4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:49.471349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 205/418] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 07:41:49 2022] PUT /identity/v3/projects/5b4d36fd27fd480aad590e2c1f973948/groups/51baed48ed134368906d54b9ce9b6e23/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:41:49.537488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d020f-8a10-40aa-897e-e82868ebed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:49.538140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d020f-8a10-40aa-897e-e82868ebed25 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.538430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d020f-8a10-40aa-897e-e82868ebed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.538706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d020f-8a10-40aa-897e-e82868ebed25 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.539449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d020f-8a10-40aa-897e-e82868ebed25 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:49.539784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d020f-8a10-40aa-897e-e82868ebed25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:49.541189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d020f-8a10-40aa-897e-e82868ebed25 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:49.556193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 214/419] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:49.562509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:49.591914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:49.704207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad2c72-3519-4b18-b8c0-1371771263c5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:49.740149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fce2f0-2fe3-4498-ae37-abf07e5d725f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.740663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fce2f0-2fe3-4498-ae37-abf07e5d725f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.740663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fce2f0-2fe3-4498-ae37-abf07e5d725f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.741970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 215/420] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:41:49.746958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221c154-99e4-4436-a36e-61371364e615 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.747245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221c154-99e4-4436-a36e-61371364e615 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.747523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221c154-99e4-4436-a36e-61371364e615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.761636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:49.761751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:49.761967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:49.762243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:49.773783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c221c154-99e4-4436-a36e-61371364e615 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:49.775302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c221c154-99e4-4436-a36e-61371364e615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:49.859694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 216/421] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:49.900235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:49.901825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221c154-99e4-4436-a36e-61371364e615 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:49.909339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:49.920172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:49.948591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:50.018320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221c154-99e4-4436-a36e-61371364e615 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:50.056093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:50.057158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.057612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.058052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.070643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:50.076682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:50.077780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:50.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:50.080759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 217/422] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:50.086741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:50.176095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:50.199980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c4edb-752c-46d9-8b8a-e1b3467a8732 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:50.222458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 206/423] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:41:50.233077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:50.242288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:50.243107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.243560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.244048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.244937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:50.245411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:50.246958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:50.256472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 218/424] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:50.261677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:50.377609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737ff69-8429-437b-a2b1-0254fb6643aa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:50.415794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:50.417347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.417846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.418332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.423984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:50.497832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:50.499903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:50.613679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 219/425] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:50.619796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8034aac-208a-4050-9bad-b08c17da9e7f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:50.620953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:50.654787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:50.677540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:50.678189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.678471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.678745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.679963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:50.681526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:50.683451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 207/426] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:50 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:50.689119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:50.695894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:50.819317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30bd35-bb14-4e77-aac1-3b3a3fe911e1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:50.873309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd50f4-4ed7-4107-a7dc-e4371791d9de None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:50.884305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe749bb-0eca-44e6-ae45-fe49aaacf421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:50.884962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe749bb-0eca-44e6-ae45-fe49aaacf421 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.885246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe749bb-0eca-44e6-ae45-fe49aaacf421 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.885523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe749bb-0eca-44e6-ae45-fe49aaacf421 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.887015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe749bb-0eca-44e6-ae45-fe49aaacf421 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:50.888548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe749bb-0eca-44e6-ae45-fe49aaacf421 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:50.890558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 208/427] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:50 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:50.899573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d54f9-0bab-4da3-ace8-09013556b76e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.899573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d54f9-0bab-4da3-ace8-09013556b76e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.899573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d54f9-0bab-4da3-ace8-09013556b76e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.899573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 209/428] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:50.899573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d54f9-0bab-4da3-ace8-09013556b76e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:50.931617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5f2aa-663a-496c-b3d8-2331e869aee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:50.932210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5f2aa-663a-496c-b3d8-2331e869aee2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.932496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5f2aa-663a-496c-b3d8-2331e869aee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.932773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5f2aa-663a-496c-b3d8-2331e869aee2 None admin] PATH_INFO: `/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.934071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5f2aa-663a-496c-b3d8-2331e869aee2 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a00a98d16a4958ba3d25aa7af0cd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:50.935683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5f2aa-663a-496c-b3d8-2331e869aee2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:50.937631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 220/429] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:50 2022] GET /identity/v3/projects/c4a00a98d16a4958ba3d25aa7af0cd5b => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:50.942532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ad2ea-74cb-4669-86dd-729a37067cff None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:50.942532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ad2ea-74cb-4669-86dd-729a37067cff None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:50.942532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ad2ea-74cb-4669-86dd-729a37067cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:50.958513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e22ad2ea-74cb-4669-86dd-729a37067cff None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:50.959738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e22ad2ea-74cb-4669-86dd-729a37067cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:50.978941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:51.033863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d54f9-0bab-4da3-ace8-09013556b76e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:51.052112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 221/430] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:51.053599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ad2ea-74cb-4669-86dd-729a37067cff None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:51.091298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:51.092616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.092794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.093069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.093861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:51.094173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:51.095643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:51.096652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:51.102618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 210/431] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:50 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:51.107653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:51.235475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ad2ea-74cb-4669-86dd-729a37067cff None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:51.290203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ce5f-8df4-4e93-aff8-8d0c94f8ab59 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:51.313203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:51.313869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.314151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.314429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.316601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:51.318505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:51.319764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 222/432] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:51 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:51.327620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:51.327873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:51.354214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:51.354416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.354774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.354885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.368049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:51.373148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:51.374024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:51.374850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 211/433] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:51.380900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:51.454773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49a3d7-045f-4f86-8307-9337359247ff None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:51.456470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:51.495076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149997d1-f596-4425-a44b-84eeab5e7a9a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:51.502044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec3661-719d-482d-a6fc-4d6f4084e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:51.502676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3661-719d-482d-a6fc-4d6f4084e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.502961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3661-719d-482d-a6fc-4d6f4084e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.503239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3661-719d-482d-a6fc-4d6f4084e3c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.504155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec3661-719d-482d-a6fc-4d6f4084e3c0 None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:51.504471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec3661-719d-482d-a6fc-4d6f4084e3c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:51.505994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec3661-719d-482d-a6fc-4d6f4084e3c0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:51.515684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 223/434] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:51 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=c4a00a98d16a4958ba3d25aa7af0cd5b&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:51.517140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee2d38-b2eb-4d29-b2ef-2f1801a7cf96 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.517436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee2d38-b2eb-4d29-b2ef-2f1801a7cf96 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.517710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee2d38-b2eb-4d29-b2ef-2f1801a7cf96 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.519381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 224/435] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:51.525204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168756f-afe9-40ff-86bc-797238573f6e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.525486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168756f-afe9-40ff-86bc-797238573f6e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.525765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168756f-afe9-40ff-86bc-797238573f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.551427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:51.552155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.552441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.552723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.553496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:51.553809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:51.555263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5168756f-afe9-40ff-86bc-797238573f6e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:51.555380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:51.557278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5168756f-afe9-40ff-86bc-797238573f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:51.566159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 212/436] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:51.572750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:51.628329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:51.663755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 225/437] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:51.698173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168756f-afe9-40ff-86bc-797238573f6e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:51.708473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927838f-1fc1-4974-bf03-cd1edf93400f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:51.748300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf392a53-44c2-41c5-8750-da5bbd9ac8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:51.749287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392a53-44c2-41c5-8750-da5bbd9ac8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.749768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392a53-44c2-41c5-8750-da5bbd9ac8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.750241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392a53-44c2-41c5-8750-da5bbd9ac8e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.834561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:51.842472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168756f-afe9-40ff-86bc-797238573f6e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:51.842779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf392a53-44c2-41c5-8750-da5bbd9ac8e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:51.845782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf392a53-44c2-41c5-8750-da5bbd9ac8e5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:51.892835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00088f74-9a17-459a-a441-b2af11a7fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:51.893487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088f74-9a17-459a-a441-b2af11a7fa56 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.893776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088f74-9a17-459a-a441-b2af11a7fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.894444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088f74-9a17-459a-a441-b2af11a7fa56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.897214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:51.910640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00088f74-9a17-459a-a441-b2af11a7fa56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-00088f74-9a17-459a-a441-b2af11a7fa56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:51.916687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:51.917599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:51.918408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 226/438] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:51.919071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b9f2c-27b2-4fd4-b8b7-3960090674dc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.919334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b9f2c-27b2-4fd4-b8b7-3960090674dc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.920245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b9f2c-27b2-4fd4-b8b7-3960090674dc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.921504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 227/439] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:51.924841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b9f2c-27b2-4fd4-b8b7-3960090674dc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:51.971178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 213/440] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:51.972479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9f942-306d-429d-a1a7-fa1bf44c88b4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:51.972769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9f942-306d-429d-a1a7-fa1bf44c88b4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:51.973043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9f942-306d-429d-a1a7-fa1bf44c88b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:51.994449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ddc9f942-306d-429d-a1a7-fa1bf44c88b4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:51.995201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ddc9f942-306d-429d-a1a7-fa1bf44c88b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:52.054412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b9f2c-27b2-4fd4-b8b7-3960090674dc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.072228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 214/441] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:52.073073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9f942-306d-429d-a1a7-fa1bf44c88b4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.089785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.090410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.090702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.090986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.091870 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:52.092279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.093692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.102410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 228/442] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:41:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:52.103204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.121794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.146581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.176965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9f942-306d-429d-a1a7-fa1bf44c88b4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.203697 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5568f5e-1491-472f-882f-a27324ffee8a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.211728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.212373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.212659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.212937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.214102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.215840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.218078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 215/443] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:52.218896 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.242683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.243322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.243636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.243913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.254079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:52.259016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:52.259966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:52.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 229/444] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:52.261454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.322562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd49fb-f259-43bc-ae20-bea5897913b3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.337273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.357289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.357903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.358184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.358451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.359158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e040ed-2f34-49ad-942e-f10b4eb2d114 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.366726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.394165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.394812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.395097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.395374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.396579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.398048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.399654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 230/445] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:52.400421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.426195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.427685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.499134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33b477-4ee1-4ebb-90a5-19d1000ababa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.528184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 216/446] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:52.532533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.533479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.534103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.534392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.534710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.535450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:52.535798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.537192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.545719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 231/447] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:52.546524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.556915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.576763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.637351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0720e-13cc-4d82-b047-27c730210f47 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.648100 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75d822-75b2-46fa-bef8-17921404e0e4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.670979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.671728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.672018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.672324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.673128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=eb040a5048fb4656b0af59ad909a2338) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:52.673476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.674910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.680634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 217/448] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:52.681991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.682633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.682921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.683198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] PATH_INFO: `/v3/projects/eb040a5048fb4656b0af59ad909a2338` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.684454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.684571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] RBAC: Authorizing `identity:get_project(project_id=eb040a5048fb4656b0af59ad909a2338)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.686429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.688461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 232/449] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/projects/eb040a5048fb4656b0af59ad909a2338 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:52.694014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.788290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.806324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9631582-2956-4ae7-b875-13a5fba26c2e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.807983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:52.814253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fde60-9ce9-48ab-aa32-8674164afda8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:52.847662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.848343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.848627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.848900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.852741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:52.853397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:52.853687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:52.853969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] PATH_INFO: `/v3/groups/89b7c81b88df4e7791116c183eccc372` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:52.855160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] RBAC: Authorizing `identity:get_group(group_id=89b7c81b88df4e7791116c183eccc372)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.856684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.859085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 233/450] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/groups/89b7c81b88df4e7791116c183eccc372 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:52.864716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:52.924126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:52.925598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:52.972636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf980e88-0b13-4275-a327-97fc9ae396a5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:53.006241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:53.006858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:53.007161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:53.007422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:53.008249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] RBAC: Adding query filter params (group.id=89b7c81b88df4e7791116c183eccc372, role.id=338a254b1f394ba2b6d9f5fb3885cc4d, scope.project.id=eb040a5048fb4656b0af59ad909a2338) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:53.008565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:53.010001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:53.016452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 234/451] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/role_assignments?group.id=89b7c81b88df4e7791116c183eccc372&scope.project.id=eb040a5048fb4656b0af59ad909a2338&role.id=338a254b1f394ba2b6d9f5fb3885cc4d => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:53.025075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 218/452] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:41:53.031679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:53.043972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:53.139699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf415-9bb7-4fc4-90d3-219943823b0d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:53.174705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:53.175350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:53.175670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:53.175953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] PATH_INFO: `/v3/projects/5b4d36fd27fd480aad590e2c1f973948` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:53.177110 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] RBAC: Authorizing `identity:get_project(project_id=5b4d36fd27fd480aad590e2c1f973948)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:53.178567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:53.180266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 235/453] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:41:53 2022] GET /identity/v3/projects/5b4d36fd27fd480aad590e2c1f973948 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:53.186551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:53.188908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:53.288797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0833b-40e4-4afd-8720-3557f7d2ec53 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:53.324121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fcfb1-ae81-47c1-aacb-4b7c805c38bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:53.324771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fcfb1-ae81-47c1-aacb-4b7c805c38bc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:53.325056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fcfb1-ae81-47c1-aacb-4b7c805c38bc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:53.325331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fcfb1-ae81-47c1-aacb-4b7c805c38bc None admin] PATH_INFO: `/v3/groups/51baed48ed134368906d54b9ce9b6e23` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:53.327058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fcfb1-ae81-47c1-aacb-4b7c805c38bc None admin] RBAC: Authorizing `identity:get_group(group_id=51baed48ed134368906d54b9ce9b6e23)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:53.328543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fcfb1-ae81-47c1-aacb-4b7c805c38bc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:53.330306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 219/454] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 07:41:53 2022] GET /identity/v3/groups/51baed48ed134368906d54b9ce9b6e23 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:53.336238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:53.417008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:53.440068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93792c1d-99c4-4282-81bb-8a4558ce6166 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:53.475978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4922b62-226b-44ca-a7c2-fea684053cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:53.476599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4922b62-226b-44ca-a7c2-fea684053cdf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:53.476885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4922b62-226b-44ca-a7c2-fea684053cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:53.477167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4922b62-226b-44ca-a7c2-fea684053cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:53.477965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4922b62-226b-44ca-a7c2-fea684053cdf None admin] RBAC: Adding query filter params (group.id=51baed48ed134368906d54b9ce9b6e23, role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=5b4d36fd27fd480aad590e2c1f973948) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:53.478280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4922b62-226b-44ca-a7c2-fea684053cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:53.479835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4922b62-226b-44ca-a7c2-fea684053cdf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:53.486287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 236/455] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 07:41:53 2022] GET /identity/v3/role_assignments?group.id=51baed48ed134368906d54b9ce9b6e23&scope.project.id=5b4d36fd27fd480aad590e2c1f973948&role.id=05421dac98a848f68d13ca21036dbfbb => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:55.885724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e4908-466a-44cc-998a-14f1ca0ddbad None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:55.886732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e4908-466a-44cc-998a-14f1ca0ddbad None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:55.887223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e4908-466a-44cc-998a-14f1ca0ddbad None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:55.889480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 220/456] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:55.895121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67629b2-c3c9-4e68-ae3d-f974f91d6e72 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:55.895558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67629b2-c3c9-4e68-ae3d-f974f91d6e72 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:55.895723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67629b2-c3c9-4e68-ae3d-f974f91d6e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:55.924537 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f67629b2-c3c9-4e68-ae3d-f974f91d6e72 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:55.925999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f67629b2-c3c9-4e68-ae3d-f974f91d6e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:56.018131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 237/457] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:56.037982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d9ae0-2352-481a-baca-65f9f909a567 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.038585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d9ae0-2352-481a-baca-65f9f909a567 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.039081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d9ae0-2352-481a-baca-65f9f909a567 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.041606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 221/458] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:56.046067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898751-67f8-49da-8332-32599668f058 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.046357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898751-67f8-49da-8332-32599668f058 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.046635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898751-67f8-49da-8332-32599668f058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.066793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-01898751-67f8-49da-8332-32599668f058 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:56.068320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-01898751-67f8-49da-8332-32599668f058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:56.074182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d9ae0-2352-481a-baca-65f9f909a567 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:56.192595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:56.199856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:56.201124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 238/459] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:56.234993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d9ae0-2352-481a-baca-65f9f909a567 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:56.237144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898751-67f8-49da-8332-32599668f058 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:56.305572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c723531-1b0c-42a4-aa10-f347700e0e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:56.306252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c723531-1b0c-42a4-aa10-f347700e0e0c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.306656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c723531-1b0c-42a4-aa10-f347700e0e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.306947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c723531-1b0c-42a4-aa10-f347700e0e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.320140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c723531-1b0c-42a4-aa10-f347700e0e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:56.322284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c723531-1b0c-42a4-aa10-f347700e0e0c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5c723531-1b0c-42a4-aa10-f347700e0e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:41:56.340226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:56.341805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:56.342681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:41:56.342681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:41:56.342681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:56.342681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:56.344550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 222/460] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:56.345944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9331c1-579e-4c2d-8410-ad42837488e2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.346367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9331c1-579e-4c2d-8410-ad42837488e2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.346778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9331c1-579e-4c2d-8410-ad42837488e2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.348175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 223/461] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:56.353443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552075ab-0ae1-466a-a6af-bfc90feb98d1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.354000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552075ab-0ae1-466a-a6af-bfc90feb98d1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.354423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552075ab-0ae1-466a-a6af-bfc90feb98d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.377282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898751-67f8-49da-8332-32599668f058 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:56.378499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-552075ab-0ae1-466a-a6af-bfc90feb98d1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:56.380988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-552075ab-0ae1-466a-a6af-bfc90feb98d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:56.414972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:56.415767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.416222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.416746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.428529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:56.430320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:41:56.447477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:56.448492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:56.449378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:41:56.449378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:41:56.449378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:56.449378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:56.452219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 239/462] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:56.453262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:56.468303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 224/463] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:41:56.469539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552075ab-0ae1-466a-a6af-bfc90feb98d1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:56.547086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:56.600661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:56.605699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552075ab-0ae1-466a-a6af-bfc90feb98d1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:56.608671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aae38-822e-410f-b0ab-b185f39d6fb4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:56.649787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:56.650513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.650866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.651206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.652170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:56.652651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:56.653866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23654337-2100-4abc-b916-dae09e1c7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:56.654185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:56.654945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23654337-2100-4abc-b916-dae09e1c7978 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.655423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23654337-2100-4abc-b916-dae09e1c7978 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.656163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23654337-2100-4abc-b916-dae09e1c7978 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.657051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23654337-2100-4abc-b916-dae09e1c7978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:56.657541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23654337-2100-4abc-b916-dae09e1c7978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:56.659850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23654337-2100-4abc-b916-dae09e1c7978 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:56.674144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 225/464] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:56.678799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:56.678917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 240/465] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:56.678917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fb18a-8de0-4b31-983c-d4ae855cc34d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.678917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fb18a-8de0-4b31-983c-d4ae855cc34d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.678917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fb18a-8de0-4b31-983c-d4ae855cc34d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.678917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 241/466] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:56.679881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fb18a-8de0-4b31-983c-d4ae855cc34d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:56.791319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f2491-6168-4a91-9c04-3b8cbe1950c1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:56.797589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fb18a-8de0-4b31-983c-d4ae855cc34d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:56.804774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:56.813071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:56.826845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:56.827585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.827885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.828167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.834835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fddda-f641-4e9e-a8bb-8071e30a78eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:56.835447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fddda-f641-4e9e-a8bb-8071e30a78eb None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.835763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fddda-f641-4e9e-a8bb-8071e30a78eb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.836042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fddda-f641-4e9e-a8bb-8071e30a78eb None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.836733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4fddda-f641-4e9e-a8bb-8071e30a78eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:56.838191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4fddda-f641-4e9e-a8bb-8071e30a78eb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:56.839697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:56.841717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:41:56.858053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:56.859147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:56.868336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:41:56.868336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:41:56.868336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:56.868336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:56.868336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 226/467] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:56.868336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:56.925768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 242/468] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:56 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:56.927239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80e328-0c02-48c0-9858-19e12f80d765 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:56.927634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80e328-0c02-48c0-9858-19e12f80d765 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:56.927976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80e328-0c02-48c0-9858-19e12f80d765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:56.971837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7b80e328-0c02-48c0-9858-19e12f80d765 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:56.973558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7b80e328-0c02-48c0-9858-19e12f80d765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:56.985533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec0791-a818-4dfe-ad33-4edaf8103a0c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:57.033038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:57.034031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.034424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.034804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.035647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:57.037257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:57.072854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 243/469] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:57.073943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80e328-0c02-48c0-9858-19e12f80d765 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:57.109884 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 227/470] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:56 2022] POST /identity/v3/users => generated 246 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:57.111043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:57.138794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:57.203607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:57.208295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80e328-0c02-48c0-9858-19e12f80d765 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:57.241039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f30e-5b2c-4215-82d2-f04cbb1082de None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:57.261792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:57.262511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.262824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.263101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.263860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:57.264171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:57.266346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:57.281663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 244/471] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:57.285447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:57.286154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.286399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.286674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.288484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:57.300686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:57.302830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:41:57.325069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:57.326552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:57.328344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:41:57.328344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:41:57.328344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:41:57.328344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:57.328344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 228/472] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:57.337636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:57.434412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bc9cc-8b7f-433e-9d25-881e71801d1b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:57.434583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:57.476967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c19ae8-64da-4e25-98b1-171473e10e52 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:57.477978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0f27d-4fce-4eee-9eee-41064d5f55b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:57.478613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0f27d-4fce-4eee-9eee-41064d5f55b4 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.478906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0f27d-4fce-4eee-9eee-41064d5f55b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.479181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0f27d-4fce-4eee-9eee-41064d5f55b4 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.479927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0f27d-4fce-4eee-9eee-41064d5f55b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:57.481386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0f27d-4fce-4eee-9eee-41064d5f55b4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:57.518265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5b201-036a-4ef3-a90d-bc18417934af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:57.519006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5b201-036a-4ef3-a90d-bc18417934af None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.519286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5b201-036a-4ef3-a90d-bc18417934af None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.519587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5b201-036a-4ef3-a90d-bc18417934af None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.520296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5b201-036a-4ef3-a90d-bc18417934af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:57.520613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5b201-036a-4ef3-a90d-bc18417934af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:57.522016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5b201-036a-4ef3-a90d-bc18417934af None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:57.539882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 229/473] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:41:57.539882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334edf1-ec18-4fff-b14e-651e64b52597 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.539882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334edf1-ec18-4fff-b14e-651e64b52597 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.539882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334edf1-ec18-4fff-b14e-651e64b52597 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.540239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 230/474] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:57.542564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334edf1-ec18-4fff-b14e-651e64b52597 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:57.583022 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 245/475] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:57 2022] POST /identity/v3/users => generated 248 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:41:57.584676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed793816-391d-4c49-af70-76b2dca81607 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.585221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed793816-391d-4c49-af70-76b2dca81607 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.585720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed793816-391d-4c49-af70-76b2dca81607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.609990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ed793816-391d-4c49-af70-76b2dca81607 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:57.611704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ed793816-391d-4c49-af70-76b2dca81607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:57.623913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:57.686613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334edf1-ec18-4fff-b14e-651e64b52597 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:57.692130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 246/476] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:57.721331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed793816-391d-4c49-af70-76b2dca81607 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:57.752936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:57.782526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:57.783181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.783469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.783817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.784532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:57.785995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:57.840138 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed793816-391d-4c49-af70-76b2dca81607 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:57.877818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:57.878451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:57.878739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:57.879214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:57.879802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:57.881264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:57.891069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 231/477] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:41:57 2022] POST /identity/v3/users => generated 249 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:41:57.931946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.sql.core [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 07:41:57.931946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 07:41:57.931946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [parameters: {'id': '5fd18942722747b690d3ad76ebec16cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 07:41:57.931946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80280) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() Aug 01 07:41:57.954544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:41:57.955483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application yield resource Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise self.value Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application session.commit() Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 07:41:57.956467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.flush() Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application compat.raise_( Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 07:41:57.958221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:41:57.959515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:41:57.961767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [parameters: {'id': '5fd18942722747b690d3ad76ebec16cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:57.963116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 07:41:57.964452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:41:57.967299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 247/478] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 07:41:57 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:41:57.967381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:58.087810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b330-6883-47e6-830c-1141d36160d4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:58.125173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf767ae-4402-47e4-9285-d9f2fe467dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:58.125827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf767ae-4402-47e4-9285-d9f2fe467dfe None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:58.126107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf767ae-4402-47e4-9285-d9f2fe467dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:58.126380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf767ae-4402-47e4-9285-d9f2fe467dfe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:58.137316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf767ae-4402-47e4-9285-d9f2fe467dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5bf767ae-4402-47e4-9285-d9f2fe467dfe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:58.142713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:58.143658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:58.144488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 232/479] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:41:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:58.149921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:58.161899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:58.330287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9e00a-d4c8-424e-802c-053947c1b1d5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:58.376801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa36331-8a21-4c1f-8a56-d3db9d9b0144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:58.377427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa36331-8a21-4c1f-8a56-d3db9d9b0144 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:58.377707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa36331-8a21-4c1f-8a56-d3db9d9b0144 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:58.377983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa36331-8a21-4c1f-8a56-d3db9d9b0144 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:58.378706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa36331-8a21-4c1f-8a56-d3db9d9b0144 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:41:58.379007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa36331-8a21-4c1f-8a56-d3db9d9b0144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:58.380476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa36331-8a21-4c1f-8a56-d3db9d9b0144 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:58.394533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 248/480] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:41:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:41:59.475908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da5c2e-c3a1-4117-96d8-fbf46c326ce4 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:59.476428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da5c2e-c3a1-4117-96d8-fbf46c326ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:59.476504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da5c2e-c3a1-4117-96d8-fbf46c326ce4 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:59.478123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 233/481] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:59.483511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad95fa-80e0-492e-ad5d-ee29e9bae237 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:59.483941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad95fa-80e0-492e-ad5d-ee29e9bae237 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:59.484114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad95fa-80e0-492e-ad5d-ee29e9bae237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:59.506786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c6ad95fa-80e0-492e-ad5d-ee29e9bae237 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:59.508043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c6ad95fa-80e0-492e-ad5d-ee29e9bae237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:59.591433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:59.598003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 249/482] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:41:59.637104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da5c2e-c3a1-4117-96d8-fbf46c326ce4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:59.721397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:59.797450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df8964-7c49-4cc6-ad44-5a6a72fd5544 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:59.797771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df8964-7c49-4cc6-ad44-5a6a72fd5544 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:59.797997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df8964-7c49-4cc6-ad44-5a6a72fd5544 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:59.799342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 250/483] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:41:59.805542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f74d-fc64-4c44-a55b-4a166fe25ec4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:59.805841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f74d-fc64-4c44-a55b-4a166fe25ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:59.806122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f74d-fc64-4c44-a55b-4a166fe25ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:59.809823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da5c2e-c3a1-4117-96d8-fbf46c326ce4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:59.837919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-84b7f74d-fc64-4c44-a55b-4a166fe25ec4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:59.839382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-84b7f74d-fc64-4c44-a55b-4a166fe25ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:59.875901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:59.876556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:59.876839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:59.877111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:59.905607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:41:59.914525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:41:59.915416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:41:59.916257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:41:59.917227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 234/484] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:41:59.929438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:59.995423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 251/485] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:00.018384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7f74d-fc64-4c44-a55b-4a166fe25ec4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.102674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f6b8f-7a49-424f-88f8-6c596eea4342 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.119911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:00.136879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7f74d-fc64-4c44-a55b-4a166fe25ec4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.161925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.162558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.162840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.163113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.163873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:00.164287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:00.165680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:00.171207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:00.180186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 235/486] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:00.181603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.182541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.182992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.183431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.186974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.224068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:00.237833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:00.238744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:00.246052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 252/487] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:00.262460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.346847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4f74-9988-4a07-9d53-7be628d762da None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.382390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51311b7-5cf6-42d1-bb6f-0e3d60c0db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.383454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51311b7-5cf6-42d1-bb6f-0e3d60c0db5b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.384018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51311b7-5cf6-42d1-bb6f-0e3d60c0db5b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.384481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51311b7-5cf6-42d1-bb6f-0e3d60c0db5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.395594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51311b7-5cf6-42d1-bb6f-0e3d60c0db5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:00.397302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51311b7-5cf6-42d1-bb6f-0e3d60c0db5b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:00.409923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8fea8-bb28-4f72-99b4-0b9b7e8af03a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b51311b7-5cf6-42d1-bb6f-0e3d60c0db5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:00.417124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:00.418284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:00.420000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:00.420000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:00.420000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:00.420000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:00.420000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 236/488] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:00.420000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c014-3344-427a-b734-f18f50477b96 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.420000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c014-3344-427a-b734-f18f50477b96 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.420377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c014-3344-427a-b734-f18f50477b96 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.425175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 237/489] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:00.425175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e3156-d692-4e51-938a-83d4dbf1390c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.425175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e3156-d692-4e51-938a-83d4dbf1390c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.425175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e3156-d692-4e51-938a-83d4dbf1390c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.425175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 238/490] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:00.425175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e3156-d692-4e51-938a-83d4dbf1390c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.453399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:00.457601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9dfa0-22ab-45e2-ae94-741767d8cc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.458220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9dfa0-22ab-45e2-ae94-741767d8cc0a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.458515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9dfa0-22ab-45e2-ae94-741767d8cc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.458776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9dfa0-22ab-45e2-ae94-741767d8cc0a None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.459476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9dfa0-22ab-45e2-ae94-741767d8cc0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:00.459914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9dfa0-22ab-45e2-ae94-741767d8cc0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:00.464345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9dfa0-22ab-45e2-ae94-741767d8cc0a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:00.474579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 253/491] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:00.474579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c8e89-d1da-4564-af7f-82bbf317746a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.474579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c8e89-d1da-4564-af7f-82bbf317746a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.474579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c8e89-d1da-4564-af7f-82bbf317746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.490452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9e9c8e89-d1da-4564-af7f-82bbf317746a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:00.491657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9e9c8e89-d1da-4564-af7f-82bbf317746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:00.541771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e3156-d692-4e51-938a-83d4dbf1390c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.546141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:00.565842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 254/492] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:00.575733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1d8e9-3989-4a1f-9105-5ac4d8125645 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.575733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1d8e9-3989-4a1f-9105-5ac4d8125645 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.575733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1d8e9-3989-4a1f-9105-5ac4d8125645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.582773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.583401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.583716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.583991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.584976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:00.585281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:00.590616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:00.591241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d1b1d8e9-3989-4a1f-9105-5ac4d8125645 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:00.591241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d1b1d8e9-3989-4a1f-9105-5ac4d8125645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:00.601713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 239/493] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:00.602639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.662940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 255/494] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:00.663826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1d8e9-3989-4a1f-9105-5ac4d8125645 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.695776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:00.715189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf144e3-cbd2-47a2-92f7-abc63e4ee5ad None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.752088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.752728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.753010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.753290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.768231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:00.769720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:00.785734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:00.786640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:00.792140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:00.792140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:00.792140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:00.792140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:00.792407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1d8e9-3989-4a1f-9105-5ac4d8125645 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.792493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 240/495] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:00.792493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.807110 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:00.828760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.829846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.830208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.830491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.842590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:00.848011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:00.848879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:00.849702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 256/496] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:00.850363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.902326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51eb71-4893-40bb-8c65-2192a79ba32f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.938347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.938994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.939282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.939584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:00.941563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:00.948326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:00.949780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:00.958704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429e8e1-491e-4aaa-8bb4-42b0e17e33e4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:00.963112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:00.964331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:42:00.965308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:00.969921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 241/497] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:00.970679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:00.997518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:00.998257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:00.998563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:00.998825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.009365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:01.015712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:01.016589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:01.017435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 257/498] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:01.018086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.049387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.075885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166745ee-6409-43b5-95a7-caef20a6b79a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.110359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.110983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.111273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.111580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.112283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:01.112593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.113986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.119885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa1a5-9c30-4044-b32e-9d3bd91aebc6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.125936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 242/499] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:01.126741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.154745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48eb7e-8c53-40fa-87b7-0f855a84a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.155396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48eb7e-8c53-40fa-87b7-0f855a84a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.155721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48eb7e-8c53-40fa-87b7-0f855a84a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.156009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48eb7e-8c53-40fa-87b7-0f855a84a0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.156726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48eb7e-8c53-40fa-87b7-0f855a84a0c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:01.157035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48eb7e-8c53-40fa-87b7-0f855a84a0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.158432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48eb7e-8c53-40fa-87b7-0f855a84a0c0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.166200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 258/500] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:01.167441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a7383-67c5-453d-9825-9c081926c91b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.167764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a7383-67c5-453d-9825-9c081926c91b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.171913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a7383-67c5-453d-9825-9c081926c91b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.173838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 259/501] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:01.174548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a7383-67c5-453d-9825-9c081926c91b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.183694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.229479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d648542-7955-4107-8d26-6d18bf11dea6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.263260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.263901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.264189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.264467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.265263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:01.265575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.267005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.269926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.277366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a7383-67c5-453d-9825-9c081926c91b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.304328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 243/502] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:01.305169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.313557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.314184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.314469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.314748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.315470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:01.315815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.317199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.325414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 260/503] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:01.325882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.406831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069106c-4b76-430a-9670-c64c67742536 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.420695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.424647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc15ea4-0c87-480f-a60f-72719407e25b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.440954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab0838-8404-4846-9400-9a6383cd4eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.441594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab0838-8404-4846-9400-9a6383cd4eeb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.441879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab0838-8404-4846-9400-9a6383cd4eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.442157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab0838-8404-4846-9400-9a6383cd4eeb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.449212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab0838-8404-4846-9400-9a6383cd4eeb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.450959 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab0838-8404-4846-9400-9a6383cd4eeb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.457886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.458526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.458816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.459098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-18ab0838-8404-4846-9400-9a6383cd4eeb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:01.463230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:01.464391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:01.465288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 244/504] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:01.467034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9d4dc-6f22-43b0-b954-a6ae939f0528 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.467034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9d4dc-6f22-43b0-b954-a6ae939f0528 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.467193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9d4dc-6f22-43b0-b954-a6ae939f0528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.471704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.473192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.485670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a7e9d4dc-6f22-43b0-b954-a6ae939f0528 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:01.486870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a7e9d4dc-6f22-43b0-b954-a6ae939f0528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:01.488330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:01.489297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:01.490375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:01.490375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:01.490375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:01.490375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:01.490375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 261/505] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:01.490375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.501969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.556552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 245/506] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:01.557405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9d4dc-6f22-43b0-b954-a6ae939f0528 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.592409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b53d5-1ee4-4974-8d8c-39c5c3367bf6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.626752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.627510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.627828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.628110 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.668594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9d4dc-6f22-43b0-b954-a6ae939f0528 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.670253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.694750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.696308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.703191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.703867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.704157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.704431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.706459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.714506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.720388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:01.730911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:01.732577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:01.733474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:01.733474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:01.733474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:01.733474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:01.733474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 246/507] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:01.733474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.800703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 262/508] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:01.801557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.836197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0c480-2a16-4c22-b1d2-bc6b649f6a1b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.869988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.870605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.870888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.871162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.872028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:01.872487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.873931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.891981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 247/509] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:01.892796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.900651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239927-b783-4f5d-88fd-7c0f8570e9a4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:01.909566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.910929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:01.934952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:01.935504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:01.935818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:01.936102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:01.936797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:01.937104 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:01.938508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:01.949273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 263/510] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:01.950576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:01.993609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a486dc6-e8a0-4530-ac30-e53e34abe661 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.027404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.028751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.029043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.029322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.039324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.040816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.051151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5f6e1-f505-470b-a446-741c2a415150 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:02.055449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:02.056329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:02.057172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:02.057172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:02.057172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:02.057172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:02.057172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 248/511] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:02.057445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.088426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.089303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.089583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.089858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.090543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:02.091411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.092830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.104179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 264/512] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:02.104956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.118888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.133731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.158216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff605b08-ec1d-49d8-8f2e-c0695643f1d9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.191580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.192189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.192475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.192752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.203701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.205176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.206800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71382694-abf6-42c1-9538-31e402bda1f3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:02.219707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:02.220678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:02.221761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:02.221761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:02.221761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:02.221761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:02.221761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 249/513] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:02.221761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.259945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.260561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.260846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.261122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.321112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac931c-e05d-43f9-975a-eae01569908a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.328076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.329555 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.337228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.354661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.355302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.355616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.355895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.356391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.363587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.365097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:02.377502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:02.378397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:02.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 250/514] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:02.380303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.429229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 265/515] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:02.430081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.480015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cdf6d-ea84-46da-9640-ff3d3f7092fe None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.514144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.514767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.515056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.515338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.516064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:02.516378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.517755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.528516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffeba7-92d3-4091-8a1c-49fe2c8715c3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.530000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 251/516] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:02.530796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.560260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.568309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.568949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.576051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.576151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.576549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.576654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.578120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:02.591404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:02.592422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 266/517] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:02.593481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.638515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a173b-39b2-47c4-b174-bbd27f4dd7ea None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.671922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.672559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.672844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.673115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.673798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:02.674099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.675454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.687048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 252/518] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:02.687858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.695193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605f39-6d1f-4815-94f3-c7fef0c379df None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.728631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.729252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.729542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.729822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.730674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:02.730943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.733212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.770768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 267/519] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:02.771635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.784853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.790579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b10d-a414-44fb-8f5e-c02544f6c20d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.819093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:02.824835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.825501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.825780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.826051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.836388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.837864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:02.853593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:02.854596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:02.855484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:02.855484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:02.855484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:02.855484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:02.855484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 253/520] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:02.855801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:02.876010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fda28-b7fc-4b0c-b109-1efaacdd4650 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.910518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.911142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.911768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.911768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.912398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.913856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:02.957552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15d656-a086-42e9-a002-952999dd64d5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:02.991244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:02.991861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:02.992147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:02.992424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:02.993218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:02.993532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:02.994980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.011482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 268/521] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:42:02 2022] POST /identity/v3/users => generated 247 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:03.011999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 254/522] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 07:42:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:03.012647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.016888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.033268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.134921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373b345-00cd-46fc-a4a7-97dded72544c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.136561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.136825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8f0ca-e683-4b6a-9d58-e605b1f47d2b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.173251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.173909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.174889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.174889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.182426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.184026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.187328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.187997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.188526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.188617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:03.197375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:03.198238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:03.199279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 269/523] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:03.200118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.271057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.272804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.281334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.315987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52652c-d3c9-4fdf-8fba-bac5a8282268 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.352902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.353622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.353986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.354331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.355109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:03.355498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.357246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.374629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 270/524] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:03.375855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.381827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 255/525] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:03.381827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.401092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.482896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139adf1e-ecf0-4e0c-88fd-1067ff75a2a2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.486375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9fde0-a8a1-463e-ae69-9fc881e678ae None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.493812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.521002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.521682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.521924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.522179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.522520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.523137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.523417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.523729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.524583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:03.525436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.529497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.552642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 256/526] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:03.552642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.602783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.604312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.629896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.663651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea582eb-c86a-4001-bf4b-9a8e3873ada0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.699339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.700937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.701378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.701801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.708946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.710554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.711354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 271/527] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:03.712225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:03.728097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:03.729168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:03.730195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:03.732047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 257/528] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:03.732962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.765110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.837517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c4b9e-c195-4671-af13-5b968fd7f0dd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.844023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7eb67-3dfb-4182-87e2-66ae163ecce4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:03.861699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:03.874122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.874961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.875271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.875556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.880737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:03.881357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:03.881643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:03.881922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:03.882714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=2f2160adab864eb9970b7767eeeb4789) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:03.883026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.884588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.886111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:03.888291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:03.899938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 258/529] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/role_assignments?user.id=2f2160adab864eb9970b7767eeeb4789&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:03.899938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:03.909625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:03.910772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:03.911778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:03.911778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:03.911778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:03.911778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:03.912410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 272/530] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:03.913404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.022340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77032786-84a9-44d8-b7dd-bb0103d78070 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.036345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:04.042173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447d0aa-3656-4634-9684-24ab383bbf47 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.064913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.065559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.075279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.075279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.079093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.080609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.083092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.083798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.084090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.084372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.085143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:04.085558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.087481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:04.101915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:04.102757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:04.103634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:04.103634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:04.103634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:04.103634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:04.103634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 259/531] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:04.104336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.107334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 273/532] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:04.113459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.151391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:04.226386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428c191-2a0d-4b42-b609-f6b788430a9a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.247482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42281-b132-49aa-8720-9a1352b5ce3b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.272021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.272652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.272943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.273470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.274258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:04.274998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.276710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.286612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.287692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.288199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.288674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.289740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:04.290268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.291001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 274/533] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:04.291927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.292574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.307026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 260/534] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:04.308383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.312108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:04.414868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:04.419601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ac772-9524-403e-bc15-4f5083d1ccdb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.421125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bda94-3c01-458f-b8c8-b412485cdd00 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.465537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.466336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.467010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.467207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.467500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.467808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.468144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.468618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=659b367f1ce643a68d55ff1c00d8b7b8) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:04.468883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.468983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.470390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.476947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.483409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 275/535] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/role_assignments?user.id=659b367f1ce643a68d55ff1c00d8b7b8&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:04.483409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.483705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:04.516739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:04.517695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:04.518625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 261/536] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:04.524541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.618072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201f2aa-b802-4029-aaa8-9dfdc3442c18 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.658311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.658965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.659252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.659658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.667287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.669493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.678219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:04.684809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:04.685770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:04.686742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:04.687616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 276/537] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:04.694318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.737861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:04.779700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37eed7-d505-477c-a96c-eb492f28b728 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.807261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d4061-a38d-4bbb-9519-ab1bf488fb54 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:04.849612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.850249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.850533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.850809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.851506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:04.851644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:04.851888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.852325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:04.852680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:04.853027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:04.853323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.853920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:04.854304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:04.856738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:04.874618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 277/538] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:04.879952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 262/539] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:04.880901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.886191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:04.972342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:05.002004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee0bb7-c121-4653-aa86-c29c4b7ce019 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:05.017131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd812a6-88ce-41f1-b380-e54a9d53f974 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:05.053581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387959b8-b554-4b53-a97c-f9001bf8f3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:05.054216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387959b8-b554-4b53-a97c-f9001bf8f3ed None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:05.054505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387959b8-b554-4b53-a97c-f9001bf8f3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:05.054828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387959b8-b554-4b53-a97c-f9001bf8f3ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:05.055675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387959b8-b554-4b53-a97c-f9001bf8f3ed None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=317df4b0baa54808af9b7918ff5c3515) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:05.056045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387959b8-b554-4b53-a97c-f9001bf8f3ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:05.057510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387959b8-b554-4b53-a97c-f9001bf8f3ed None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:05.063961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 278/540] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/role_assignments?user.id=317df4b0baa54808af9b7918ff5c3515&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:05.079663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae4577-cb50-483c-9f2c-e360249da0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:05.080475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae4577-cb50-483c-9f2c-e360249da0ae None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:05.080865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae4577-cb50-483c-9f2c-e360249da0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:05.081152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae4577-cb50-483c-9f2c-e360249da0ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:05.082003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae4577-cb50-483c-9f2c-e360249da0ae None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=ad3cf8a3e0a346fa867445d6b6300447) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:05.082322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae4577-cb50-483c-9f2c-e360249da0ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:05.083879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae4577-cb50-483c-9f2c-e360249da0ae None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:05.087339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:05.092383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 263/541] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:04 2022] GET /identity/v3/role_assignments?user.id=ad3cf8a3e0a346fa867445d6b6300447&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:05.319428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23baac4-c953-4eb9-b1f6-fa708691d2ae None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:05.319706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23baac4-c953-4eb9-b1f6-fa708691d2ae None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:05.319954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23baac4-c953-4eb9-b1f6-fa708691d2ae None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:05.321466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 279/542] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:05.327761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b56d3-99f3-4052-bc93-92cfefa670ef None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:05.328061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b56d3-99f3-4052-bc93-92cfefa670ef None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:05.328346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b56d3-99f3-4052-bc93-92cfefa670ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:05.347889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bc1b56d3-99f3-4052-bc93-92cfefa670ef None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:05.349096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bc1b56d3-99f3-4052-bc93-92cfefa670ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:05.431573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 264/543] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:05.468410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23baac4-c953-4eb9-b1f6-fa708691d2ae None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:05.596366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23baac4-c953-4eb9-b1f6-fa708691d2ae None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:05.641779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:05.642541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:05.642723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:05.643007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:05.656378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:05.661885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:05.663941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:05.671465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 280/544] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:05.673122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b56d3-99f3-4052-bc93-92cfefa670ef None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:05.845709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b56d3-99f3-4052-bc93-92cfefa670ef None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:05.879901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:05.895694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:05.907090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:05.907090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:05.907090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:05.907090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:05.907090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:05.907090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:05.908662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 265/545] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:05.914272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:05.917157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:06.047791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83bcdc-deb3-4c0c-8ee3-3c723e93e086 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:06.084109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:06.084758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.085044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.085326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:06.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:06.098140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:06.116034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:06.116919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:06.118323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:06.118323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:06.118323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:06.118323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:06.118323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 281/546] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:06.125383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:06.242398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b99be-bdcf-41eb-aeb8-0613324d0cc4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:06.283112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd61c8b-629d-4732-b0e2-25ba458a7b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:06.283887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd61c8b-629d-4732-b0e2-25ba458a7b5d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.284174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd61c8b-629d-4732-b0e2-25ba458a7b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.284446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd61c8b-629d-4732-b0e2-25ba458a7b5d None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:06.285143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd61c8b-629d-4732-b0e2-25ba458a7b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:06.285448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd61c8b-629d-4732-b0e2-25ba458a7b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:06.286842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd61c8b-629d-4732-b0e2-25ba458a7b5d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:06.300591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 266/547] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:06.305937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:06.355114 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bee01-504f-46f6-ba9e-2a12719aa078 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.355424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bee01-504f-46f6-ba9e-2a12719aa078 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.355686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bee01-504f-46f6-ba9e-2a12719aa078 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:06.357662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 267/548] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:06.369115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f18af5-3475-4401-a61d-d977b77fc021 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.369404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f18af5-3475-4401-a61d-d977b77fc021 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.369999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f18af5-3475-4401-a61d-d977b77fc021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:06.405213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b7f18af5-3475-4401-a61d-d977b77fc021 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:06.406440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b7f18af5-3475-4401-a61d-d977b77fc021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:06.419879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:06.470955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080ff43-f032-4022-abc0-5353aa9437cb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:06.473035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:06.509316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:06.509886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.510173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.510449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:06.524084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:06.526919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:06.549417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:06.552090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:06.554273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 282/549] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:06.559779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 268/550] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:06.562151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:06.593212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f18af5-3475-4401-a61d-d977b77fc021 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:06.721981 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d93ff4-bae5-48db-b6f7-91920cd85490 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:06.738951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:06.773333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:06.774191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.774626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.776380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:06.778832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:06.780199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:06.783283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:06.790467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f18af5-3475-4401-a61d-d977b77fc021 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:06.821340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 283/551] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 07:42:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:06.832552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:06.858801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:06.859443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.859745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.860014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:06.877366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:06.882414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:06.883547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:06.884819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 269/552] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:06.890200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:06.952874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:06.958209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200901b6-b219-4c59-a670-8f519022ca75 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:06.995900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506be076-6897-48a9-8af3-e7e2504a7539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:06.997435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506be076-6897-48a9-8af3-e7e2504a7539 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:06.997720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506be076-6897-48a9-8af3-e7e2504a7539 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:06.997998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506be076-6897-48a9-8af3-e7e2504a7539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.016617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99086-ee9b-4469-8234-035a2aa02122 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.055386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d46c6-8b61-4e49-abd0-2e6a396f5863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.056043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d46c6-8b61-4e49-abd0-2e6a396f5863 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.056319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d46c6-8b61-4e49-abd0-2e6a396f5863 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.056590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d46c6-8b61-4e49-abd0-2e6a396f5863 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.057304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d46c6-8b61-4e49-abd0-2e6a396f5863 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:07.057607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d46c6-8b61-4e49-abd0-2e6a396f5863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.057774 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.059006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d46c6-8b61-4e49-abd0-2e6a396f5863 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.068151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 270/553] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:07.069451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043660e-b8e7-4fa6-9c16-3a6f72cbfd3d None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.069737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043660e-b8e7-4fa6-9c16-3a6f72cbfd3d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.070009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043660e-b8e7-4fa6-9c16-3a6f72cbfd3d None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.071442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 271/554] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:07.076670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043660e-b8e7-4fa6-9c16-3a6f72cbfd3d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.092385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506be076-6897-48a9-8af3-e7e2504a7539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.094657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506be076-6897-48a9-8af3-e7e2504a7539 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.190716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.204853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043660e-b8e7-4fa6-9c16-3a6f72cbfd3d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.214150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 284/555] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:07.215933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b64fc-df48-4017-bad7-e844be5cd1fe None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.216223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b64fc-df48-4017-bad7-e844be5cd1fe None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.216543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b64fc-df48-4017-bad7-e844be5cd1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.236210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a48b64fc-df48-4017-bad7-e844be5cd1fe None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:07.237389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a48b64fc-df48-4017-bad7-e844be5cd1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:07.247757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.248420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.248706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.248984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.262723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.264788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:07.281610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:07.283894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:07.286017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:07.286017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:07.286017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:07.286017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:07.286017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 272/556] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:07.286017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.326907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 285/557] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:07.326907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b64fc-df48-4017-bad7-e844be5cd1fe None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.329076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.413914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e21d75-a6e3-43b0-b7b6-f47d6248d1ab None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.449753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.450399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.450679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.450956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.456597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b64fc-df48-4017-bad7-e844be5cd1fe None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.462542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.464392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.467065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:07.481798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:07.483370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:07.484753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:07.484753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:07.484753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:07.484753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:07.486128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 273/558] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:07.489599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.500354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.501361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.501915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.502197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.502909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:07.503218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.505382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.522316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 286/559] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:07.523342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.597733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.602688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e6862-8cde-45b1-8ae4-b274bfa3a228 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.636611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34535df7-4f4b-4858-bdc0-1af9babee9e2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.639499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.640173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.640464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.640745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.652390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:07.657853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:07.659509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 274/560] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:07.660534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.678757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de092d-a7da-4789-a79a-6180b2e1620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.679375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de092d-a7da-4789-a79a-6180b2e1620d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.679691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de092d-a7da-4789-a79a-6180b2e1620d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.679965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de092d-a7da-4789-a79a-6180b2e1620d None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.680667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de092d-a7da-4789-a79a-6180b2e1620d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:07.680960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de092d-a7da-4789-a79a-6180b2e1620d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.682341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de092d-a7da-4789-a79a-6180b2e1620d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.694801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 287/561] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:07.695993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29544dd-c1cb-41d5-b26d-92e8818c693d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.696282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29544dd-c1cb-41d5-b26d-92e8818c693d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.696559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29544dd-c1cb-41d5-b26d-92e8818c693d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.699666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 288/562] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:07.699666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453964-33a9-4e10-961d-88fcba8881b2 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.699666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453964-33a9-4e10-961d-88fcba8881b2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.700419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453964-33a9-4e10-961d-88fcba8881b2 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.701749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 289/563] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:07.703935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453964-33a9-4e10-961d-88fcba8881b2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.716991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.767931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb934da-b3e9-4e12-a645-7781ed9de636 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.802249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.802889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.803180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.803458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.806778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453964-33a9-4e10-961d-88fcba8881b2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.811059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.812633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:07.823897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:07.825395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:07.826365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 275/564] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:07.827194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.832583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.846001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29a96-6613-4d4b-94af-31b9ee690fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.846617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29a96-6613-4d4b-94af-31b9ee690fac None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.846898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29a96-6613-4d4b-94af-31b9ee690fac None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.847170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29a96-6613-4d4b-94af-31b9ee690fac None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.847977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29a96-6613-4d4b-94af-31b9ee690fac None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:07.848230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29a96-6613-4d4b-94af-31b9ee690fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.849597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29a96-6613-4d4b-94af-31b9ee690fac None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.858757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 290/565] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:07.859975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa522-651d-4541-b6a6-5a6df10389ca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.860265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa522-651d-4541-b6a6-5a6df10389ca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.860549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa522-651d-4541-b6a6-5a6df10389ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.878906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c67aa522-651d-4541-b6a6-5a6df10389ca None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:07.880138 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c67aa522-651d-4541-b6a6-5a6df10389ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:07.932592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c776a-c0fb-4a01-b32e-fe46b2424896 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.945576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 291/566] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:07.946794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1355b-1a45-4c01-8792-33949e11ea4c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.947078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1355b-1a45-4c01-8792-33949e11ea4c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.947354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1355b-1a45-4c01-8792-33949e11ea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.954481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.966237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f8a1355b-1a45-4c01-8792-33949e11ea4c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:07.968070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f8a1355b-1a45-4c01-8792-33949e11ea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:07.975300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.976113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.976595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.977021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.984715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.986171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:07.998030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:07.998935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 276/567] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:08.000338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.038680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 292/568] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:08.039540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1355b-1a45-4c01-8792-33949e11ea4c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.088115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.106912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3369b-3f4a-40f1-bce2-ff780d1ac915 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.138522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.139151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.139425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.139727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.140520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:08.140828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.142268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.142586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1355b-1a45-4c01-8792-33949e11ea4c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.159679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 277/569] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:08.160484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.182938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.183605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.184035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.184320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.191057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.194806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.196329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:08.211065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:08.212133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:08.213071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:08.213071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:08.213071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:08.213071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:08.213071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 293/570] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:08.213071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.280839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709d35c-ec53-4689-ace0-ebda934164d0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.310650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.314999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.315682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.315972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.316251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.316367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795c82a-726a-432e-aac1-81bdbe7d5a9e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.326432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:08.331370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:08.332338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:08.333201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 278/571] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:08.333913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.356879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.357561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.357793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.358074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.358831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:08.359161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.360638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.372660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 294/572] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:08.373450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.429980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.438559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62ba52-3591-41cd-8b44-f307a157cfad None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.472693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250433b-9230-4f11-aa79-eead0a9bbec9 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.474390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.475042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.475328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.475653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.486524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:08.491476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:08.492604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 279/573] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:08.493474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.509890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.510524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.510807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.511079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.521307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.522769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:08.537662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:08.538596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:08.539508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:08.539508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:08.539508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:08.539508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:08.539508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 295/574] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:08.539880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.549830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.598603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7a325-45e9-498f-bcb6-9f23d65b691a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.631327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.631971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.632251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.632525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.633202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:08.633506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.634864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.640887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53eec7-4246-4a90-b2c7-456c48ba7782 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.646233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 280/575] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:08.647018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.665470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.675498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.680154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.680154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.680154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.680154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:08.680154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.680154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.687335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 296/576] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:08.688153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.752068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db4cbf-97b6-4049-92ca-fcccaca66883 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.760488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.788498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.789117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.789407 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.789680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.790466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=262558e0503a4f57a619c75acfdae331) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:08.790776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.792253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.792795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786435e-b470-4602-b4b6-f7f5f4c52b76 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.797651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 281/577] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/role_assignments?user.id=262558e0503a4f57a619c75acfdae331&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:08.798441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.831837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.832479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.833398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.833398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.833886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:08.834199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.835655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.845567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 297/578] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:08.846799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:08.903128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:08.927164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d6385-2058-432c-8d23-0a2b337067ee None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.971129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:08.971786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5edf5c-f887-4654-87f3-dfa35dbc323e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:08.971870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:08.972155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:08.972428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:08.973295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:08.973606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:08.975838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:08.989108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 282/579] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:08.989955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.008464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.014117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.015143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.015143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.015367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.023007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.024508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:09.035907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:09.036996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:09.037960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:09.039838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 298/580] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:09.041033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.102948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8adcb6-00e0-4bf6-b4c9-85dd8b6daf0c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.142016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.142762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.143055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.143332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.146504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.147739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae7d74-7fb8-4554-a176-8bd35e9736fe None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.154557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.156069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:09.171400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:09.172522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:09.173695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:09.173695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:09.173695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:09.173695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:09.173695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 283/581] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:09.173695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.191361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.192278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.192724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.193008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.204703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.206170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:09.221122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:09.222077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:09.222983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:09.222983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:09.222983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:09.222983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:09.222983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 299/582] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:09.223270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.282081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.308613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbed04-9bb7-44e9-b286-fb90b9fde757 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.341061 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123be45-660f-48c9-abdd-24ab0dd3428b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.346278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.347100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.347559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.348137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.356332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.357884 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:09.371437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:09.372979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 284/583] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:09.373887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.382009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.382635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.382921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.383196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.384449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:09.384813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.386654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.410083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 300/584] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:09.410083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.434918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.491892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e387558-c84e-45dd-ac52-cd603c2cdddc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.515426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c35066-f9d6-4dab-911e-7304bafc307e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.517206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.530234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.530871 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.531160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.531440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.533203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:09.533203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.534059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.546363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 285/585] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:09.547246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.562367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.563002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.563292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.563721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.564429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:09.564735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.566129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.578019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 301/586] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:09.578812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.658576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf818fb-9013-416d-8c1e-f3bc69f018c8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.669301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.685698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114ca0a-5e95-44d9-a51f-3617b7b1e6fd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.697745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58306038-6008-411c-8bb7-56085f33330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.698802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58306038-6008-411c-8bb7-56085f33330c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.699088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58306038-6008-411c-8bb7-56085f33330c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.699361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58306038-6008-411c-8bb7-56085f33330c None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.700454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58306038-6008-411c-8bb7-56085f33330c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:09.700830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58306038-6008-411c-8bb7-56085f33330c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.702263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58306038-6008-411c-8bb7-56085f33330c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.714804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 286/587] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:09.716019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58306038-6008-411c-8bb7-56085f33330c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.725598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.726248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.726541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.726822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.738080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.739567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:09.754698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:09.755842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:09.756759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:09.756759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:09.756759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:09.756759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:09.756759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 302/588] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:09.756759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.773993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.825557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58306038-6008-411c-8bb7-56085f33330c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.861326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.861971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.862257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.862532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.862953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e79fb-fcbe-4269-b5da-e7a24842c33d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.870793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.872940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:09.885139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:09.886312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:09.887927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 287/589] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:09.888782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.905082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.905741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.906030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.906309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.915305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.917011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.921760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:09.929386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:09.937475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 303/590] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:09.938559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.000267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79ee7-cab7-459d-8324-13ee8d468b06 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.023278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:10.035963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.036643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.036922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.039441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/2f2160adab864eb9970b7767eeeb4789/roles/49afe9a7e8ab49dca85e0aa074a1ee33` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.049806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab0ef3-3e1e-46a2-b501-cd9f57de05cc None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.053737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=2f2160adab864eb9970b7767eeeb4789, role_id=49afe9a7e8ab49dca85e0aa074a1ee33)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.055192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.084152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.084758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.085043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.085320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.086011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:10.086319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.087724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.099336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 304/591] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:10.100155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.127559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 288/592] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:09 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/2f2160adab864eb9970b7767eeeb4789/roles/49afe9a7e8ab49dca85e0aa074a1ee33 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:10.127559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.233257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:10.238139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8889687-dd00-4016-948e-da2292c621cb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.258277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0739-d40f-43df-a73e-85cffbb88dfb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.281265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.282058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.282883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.282883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.284130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:10.284549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.286192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.288035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:10.306478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 305/593] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:10.307327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.314341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.314832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.315172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.315450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.316806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:10.317120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.320549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.342933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 289/594] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:10.343896 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.449841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3583a-3c0c-420c-8c74-405edcf68adf None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.450660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c98cf-575e-473e-8eda-f1d23d971e82 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.496369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.497920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.498365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.499053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.506073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.506925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.507174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.507552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.508572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.510892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:10.512005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.519022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.520784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:10.526645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:10.528255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:10.529511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:10.532440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 306/595] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:10.533450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:10.544041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:10.545250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:10.546353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:10.546353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:10.546353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:10.546353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:10.546353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 290/596] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:10.547106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.609710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:10.660690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa078-33bd-4bb9-830b-ab79bf3b4c85 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.677009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888d3f-8a36-4b7c-96cf-515f41a12499 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.703254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.704460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.704901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.705361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.706904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:10.707378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.709500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.718440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.719235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.719653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.720407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.726450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 291/597] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:10.727461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.736708 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.738350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:10.755461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:10.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:10.757758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:10.757758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:10.757758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:10.757758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:10.760862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 307/598] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:10.760862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:10.814330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:10.869763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00dd5c-7036-4eab-9cf9-c1188a8283ce None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.872245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db02823-6366-408d-9b4a-f9ab3840ad14 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:10.875426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:10.909495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37251337-8054-425e-8fc6-ad381e49d94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.910156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37251337-8054-425e-8fc6-ad381e49d94b None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.910449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37251337-8054-425e-8fc6-ad381e49d94b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.910732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37251337-8054-425e-8fc6-ad381e49d94b None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/659b367f1ce643a68d55ff1c00d8b7b8/roles/49afe9a7e8ab49dca85e0aa074a1ee33` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.911022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:10.911885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:10.912351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:10.912821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:10.914659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:10.915179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.916773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.926607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37251337-8054-425e-8fc6-ad381e49d94b None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=659b367f1ce643a68d55ff1c00d8b7b8, role_id=49afe9a7e8ab49dca85e0aa074a1ee33)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:10.928122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37251337-8054-425e-8fc6-ad381e49d94b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:10.934685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 292/599] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:10.935703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.000325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 308/600] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:10 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/659b367f1ce643a68d55ff1c00d8b7b8/roles/49afe9a7e8ab49dca85e0aa074a1ee33 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:11.001914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a178e0-eb5b-46d9-ba1e-20150b1632f5 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.002409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a178e0-eb5b-46d9-ba1e-20150b1632f5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.002908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a178e0-eb5b-46d9-ba1e-20150b1632f5 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.004854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 309/601] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:11.009767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a178e0-eb5b-46d9-ba1e-20150b1632f5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.052166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14460085-584f-46dd-976b-fd08c96b86ed None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.066837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:11.104396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708203c-afa5-449b-a451-992f22e5afd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.105048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708203c-afa5-449b-a451-992f22e5afd8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.105333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708203c-afa5-449b-a451-992f22e5afd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.105695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708203c-afa5-449b-a451-992f22e5afd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.107082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708203c-afa5-449b-a451-992f22e5afd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:11.107391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708203c-afa5-449b-a451-992f22e5afd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:11.108837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708203c-afa5-449b-a451-992f22e5afd8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:11.127622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 293/602] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:11.129063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb807034-3b67-41ab-9a0d-68c4c1cd629c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.129604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb807034-3b67-41ab-9a0d-68c4c1cd629c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.129879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb807034-3b67-41ab-9a0d-68c4c1cd629c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.146156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a178e0-eb5b-46d9-ba1e-20150b1632f5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.157320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bb807034-3b67-41ab-9a0d-68c4c1cd629c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:11.158590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bb807034-3b67-41ab-9a0d-68c4c1cd629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:11.196906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.197794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.198222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.198664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.206753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:11.208289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:11.220448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:11.221594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:11.222735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 310/603] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:11.223342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.244738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 294/604] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:11.245611 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb807034-3b67-41ab-9a0d-68c4c1cd629c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.249560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:11.362568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3faea-37fc-439b-ba03-75bbd789fbf1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.386651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:11.392667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb807034-3b67-41ab-9a0d-68c4c1cd629c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.399244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.399923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.400207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.400481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.408915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:11.410376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:11.423582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:11.424700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:11.425723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 311/605] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:11.426276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.441937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.443072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.443505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.444203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.445486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:11.445947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:11.448090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:11.466576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 295/606] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:11.467726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.506096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:11.534538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773e500-5d2d-4f89-8fce-a97462e34760 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.576647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.577822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.578258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.579107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.594542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:11.597960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115aa94-72fd-40ae-86de-372fee6d6f08 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:11.602152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:11.603255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:11.604261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:11.606991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 312/607] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:11.607934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.640529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.641114 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.641347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.641621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.642581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:11.642697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:11.644246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:11.658106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 296/608] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:11.659041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.680709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:11.739522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500a3d-6ae1-4592-93bf-b4ae32671bc7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.779489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.780519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.780902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.781539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/ad3cf8a3e0a346fa867445d6b6300447/roles/49afe9a7e8ab49dca85e0aa074a1ee33` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.785364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:11.789945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74866399-d94a-44ec-85de-f4550bd55465 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:11.803124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=ad3cf8a3e0a346fa867445d6b6300447, role_id=49afe9a7e8ab49dca85e0aa074a1ee33)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:11.805759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:11.831892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:11.832563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:11.832853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:11.833129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:11.833846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:11.834152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:11.835547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:11.849992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 297/609] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:11.849992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.916480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 313/610] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:11 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/ad3cf8a3e0a346fa867445d6b6300447/roles/49afe9a7e8ab49dca85e0aa074a1ee33 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:11.917000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:11.969074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:12.014121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75230938-ea76-4adf-9d44-4d300e9788dd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.096034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:12.096802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.097009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.097291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/317df4b0baa54808af9b7918ff5c3515/roles/49afe9a7e8ab49dca85e0aa074a1ee33` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.122847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=317df4b0baa54808af9b7918ff5c3515, role_id=49afe9a7e8ab49dca85e0aa074a1ee33)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:12.124051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a712e6-c47b-49fc-b2bb-d8c0f9201b5e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.124350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:12.165652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451598a-3255-40d4-b9a1-d8728253ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:12.166791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451598a-3255-40d4-b9a1-d8728253ce24 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.167280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451598a-3255-40d4-b9a1-d8728253ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.167783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451598a-3255-40d4-b9a1-d8728253ce24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.179872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451598a-3255-40d4-b9a1-d8728253ce24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:12.181666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451598a-3255-40d4-b9a1-d8728253ce24 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e451598a-3255-40d4-b9a1-d8728253ce24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:12.206791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:12.207876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:12.208834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:12.208834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:12.208834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:12.208834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:12.217956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 298/611] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:11 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/317df4b0baa54808af9b7918ff5c3515/roles/49afe9a7e8ab49dca85e0aa074a1ee33 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:12.218630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 314/612] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:12.220348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:12.277446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404805-dfb9-42d2-ab38-5dded6555e71 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.278206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404805-dfb9-42d2-ab38-5dded6555e71 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.278694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404805-dfb9-42d2-ab38-5dded6555e71 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.280777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 315/613] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:12.286231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a49b93-156c-4ab6-b7fd-5a53ee1b995a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.286827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a49b93-156c-4ab6-b7fd-5a53ee1b995a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.287306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a49b93-156c-4ab6-b7fd-5a53ee1b995a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.316007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c5a49b93-156c-4ab6-b7fd-5a53ee1b995a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:12.318145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c5a49b93-156c-4ab6-b7fd-5a53ee1b995a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:12.391852 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:12.418551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13131ad-1e1a-4d1e-999d-903ae1950ff8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.446050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 316/614] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:12.449122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:12.455738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:12.456546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.457004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.457425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.458268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:12.458721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:12.460268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:12.473662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 299/615] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:12.474048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a49b93-156c-4ab6-b7fd-5a53ee1b995a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:12.481369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:12.583156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a49b93-156c-4ab6-b7fd-5a53ee1b995a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.588599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54b80f-c88c-4862-987f-d871917f7a5b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.625888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:12.626816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.627573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.628003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.632799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:12.633586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.633972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.634352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.637816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:12.639438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:12.651716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:12.652787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:12.653737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:12.659875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 300/616] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:12.661421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:12.673255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:12.678018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:12.679141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:12.680202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:12.687988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 317/617] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:12.696536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:12.723941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:12.747781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:12.774337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e3ad0-8981-400f-a5e1-55aefe842c42 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.814573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:12.815574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.816008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.816582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.817525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:12.818009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:12.819738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:12.840612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 301/618] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 07:42:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:12.846319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:12.851337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff517755-88d7-4e69-96d9-aa905b2b7061 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.887735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:12.888368 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.888660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.888941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.889670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:12.889980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:12.891382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:12.900331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 318/619] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:12.908543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:12.957449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27bda-f440-4579-9f08-7629afab29c6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:12.975948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:12.999901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.000878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.001371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.001822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.007906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:13.023920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823d805-8d3c-464f-bb55-39cc85c2cf19 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.031652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.033141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:13.066140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:13.067143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:13.068073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:13.068073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:13.068073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:13.068073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:13.068306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.068394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 302/620] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:13.068477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.068854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.068975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.074877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.080486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.081976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:13.097398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:13.098312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:13.100171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:13.100171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:13.100171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:13.100171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:13.100171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 319/621] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:13.106656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.262484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a074d43-2b31-42b7-ba64-7f3c791413b6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.300575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.301230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.301513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.301787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.302493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:13.302827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.307554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.318461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 303/622] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:13.319446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ecbfc-7ede-4b5b-869c-53098946740e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.331486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.358479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.359121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.359386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.359706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.360406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:13.360704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.362732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.364367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:13.372599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:13.379110 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 320/623] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:13.392686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.451215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5d8c0-8ce5-44ca-8cc1-19be513369c4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.492221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadd101-0c98-4c6f-9ef1-c7e782c92940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.492919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadd101-0c98-4c6f-9ef1-c7e782c92940 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.493207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadd101-0c98-4c6f-9ef1-c7e782c92940 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.493480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadd101-0c98-4c6f-9ef1-c7e782c92940 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.503257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadd101-0c98-4c6f-9ef1-c7e782c92940 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.504732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadd101-0c98-4c6f-9ef1-c7e782c92940 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-daadd101-0c98-4c6f-9ef1-c7e782c92940 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:13.520977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:13.521949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:13.522870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:13.527146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 304/624] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:13.529417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d746-9522-4ef9-8a14-9dcd321e6596 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.529719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d746-9522-4ef9-8a14-9dcd321e6596 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.530095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d746-9522-4ef9-8a14-9dcd321e6596 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.532041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 305/625] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:13.533132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5d746-9522-4ef9-8a14-9dcd321e6596 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.539783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058a825-3015-4b6a-906a-c715c8d00e4d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.582176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf66e6-274b-4787-98be-8e55ee24343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.583175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf66e6-274b-4787-98be-8e55ee24343b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.583830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf66e6-274b-4787-98be-8e55ee24343b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.584316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf66e6-274b-4787-98be-8e55ee24343b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.594064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf66e6-274b-4787-98be-8e55ee24343b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.595907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf66e6-274b-4787-98be-8e55ee24343b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2fcf66e6-274b-4787-98be-8e55ee24343b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:13.608397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:13.609402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:42:13.610357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:13.614999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 321/626] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:13.616589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a8d7-4b38-4afc-bf4a-59c2b7c6a34a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.617098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a8d7-4b38-4afc-bf4a-59c2b7c6a34a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.617579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a8d7-4b38-4afc-bf4a-59c2b7c6a34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.638708 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fd11a8d7-4b38-4afc-bf4a-59c2b7c6a34a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:13.640377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fd11a8d7-4b38-4afc-bf4a-59c2b7c6a34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:13.641683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:13.667725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5d746-9522-4ef9-8a14-9dcd321e6596 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.706963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:13.717830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.718547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.718830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.719103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.719879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:13.720190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.721183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 322/627] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:13.721727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.731075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11a8d7-4b38-4afc-bf4a-59c2b7c6a34a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.737143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 306/628] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:13.747560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.846093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11a8d7-4b38-4afc-bf4a-59c2b7c6a34a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.849467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472690f0-3118-4646-bb7b-d87ea886e714 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:13.883234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.884429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.884539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:13.885076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.885196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:13.885466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:13.885751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/262558e0503a4f57a619c75acfdae331/roles/49afe9a7e8ab49dca85e0aa074a1ee33` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.886139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:13.887972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:13.888538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.890258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.897869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:13.902319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=262558e0503a4f57a619c75acfdae331, role_id=49afe9a7e8ab49dca85e0aa074a1ee33)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:13.904512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:13.909999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 323/629] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:13.913297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:13.963008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:13.995705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 307/630] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:13 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/262558e0503a4f57a619c75acfdae331/roles/49afe9a7e8ab49dca85e0aa074a1ee33 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:13.999987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.049718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807fe82-8114-4515-9260-d8ec5d934a01 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.125527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.126194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.126479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.126756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.147162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:14.155447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:14.156525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:14.157375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:14.166894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 324/631] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:14.173183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.214602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86124680-b656-487a-b528-c8b47d3627f9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.259489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d07e2-1877-4b6c-886e-a30a669a94ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.260281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d07e2-1877-4b6c-886e-a30a669a94ec None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.260590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d07e2-1877-4b6c-886e-a30a669a94ec None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.261098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d07e2-1877-4b6c-886e-a30a669a94ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.322060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257cc3-0b07-4b0f-b13e-f44d631173fe None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.330679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:14.336849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:14.342553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d07e2-1877-4b6c-886e-a30a669a94ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:14.344061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d07e2-1877-4b6c-886e-a30a669a94ec None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:14.372091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f011ac-0dc8-49cd-b364-994e9ba61421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.372751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f011ac-0dc8-49cd-b364-994e9ba61421 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.373041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f011ac-0dc8-49cd-b364-994e9ba61421 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.373315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f011ac-0dc8-49cd-b364-994e9ba61421 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.374045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f011ac-0dc8-49cd-b364-994e9ba61421 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:14.374354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f011ac-0dc8-49cd-b364-994e9ba61421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:14.375835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f011ac-0dc8-49cd-b364-994e9ba61421 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:14.384482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 325/632] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:14.389421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f338b-3860-47a1-b978-1ad99744a095 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.389421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f338b-3860-47a1-b978-1ad99744a095 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.389421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f338b-3860-47a1-b978-1ad99744a095 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.389421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 326/633] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:14.392990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f338b-3860-47a1-b978-1ad99744a095 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.477614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 308/634] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:14.479999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e743dd8-4dc8-41ff-a587-7df8920bf89c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.480797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e743dd8-4dc8-41ff-a587-7df8920bf89c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.481091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e743dd8-4dc8-41ff-a587-7df8920bf89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.500418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1e743dd8-4dc8-41ff-a587-7df8920bf89c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:14.501665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e743dd8-4dc8-41ff-a587-7df8920bf89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:14.518580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f338b-3860-47a1-b978-1ad99744a095 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.556701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.557384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.557719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.558005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.571245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:14.572864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:14.587309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 309/635] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:14.588779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e743dd8-4dc8-41ff-a587-7df8920bf89c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:14.589267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:14.590361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:14.591644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:14.591644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:14.591644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:14.591644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:14.591644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 327/636] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:14.596432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.623935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:14.631004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:14.705053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e743dd8-4dc8-41ff-a587-7df8920bf89c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.715285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f5f84-95cb-424f-b197-456b0c3354d1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.745357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d7ce-0482-454a-be87-35f216c029e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.747132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d7ce-0482-454a-be87-35f216c029e0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.747660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d7ce-0482-454a-be87-35f216c029e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.748156 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d7ce-0482-454a-be87-35f216c029e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.750258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.750876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.751622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.751909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.752611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:14.752923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:14.754328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:14.763651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d7ce-0482-454a-be87-35f216c029e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:14.765237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d7ce-0482-454a-be87-35f216c029e0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:14.775987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 328/637] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:14.775987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-db56d7ce-0482-454a-be87-35f216c029e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:14.785350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:14.786488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:14.787631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:14.787631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:14.787631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:14.787631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:14.788700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 310/638] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:14.789935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8728d-7e01-4676-9477-789768284162 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.790281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8728d-7e01-4676-9477-789768284162 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.790629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8728d-7e01-4676-9477-789768284162 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.792089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 311/639] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:14.793524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8728d-7e01-4676-9477-789768284162 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.872746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:14.873586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:14.892561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111911af-c885-48fb-9c58-a8db19cbebb7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.905105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8728d-7e01-4676-9477-789768284162 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:14.930672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.931327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.931650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.931935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.942711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:14.945579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e016011-3563-4b9d-a8fb-e0def28dcdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:14.946301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016011-3563-4b9d-a8fb-e0def28dcdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.946656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016011-3563-4b9d-a8fb-e0def28dcdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.947002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016011-3563-4b9d-a8fb-e0def28dcdd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:14.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:14.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 329/640] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:14.950263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:14.958600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e016011-3563-4b9d-a8fb-e0def28dcdd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:14.960214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e016011-3563-4b9d-a8fb-e0def28dcdd9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7e016011-3563-4b9d-a8fb-e0def28dcdd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:14.973266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:14.979101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 312/641] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:42:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561247c-cad0-4c4a-82c7-36b581e6eca6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:14.980147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561247c-cad0-4c4a-82c7-36b581e6eca6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:14.981095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561247c-cad0-4c4a-82c7-36b581e6eca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:14.994858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6561247c-cad0-4c4a-82c7-36b581e6eca6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:14.996125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6561247c-cad0-4c4a-82c7-36b581e6eca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:15.065924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 313/642] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:15.067000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561247c-cad0-4c4a-82c7-36b581e6eca6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.067194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23693-df03-49dd-b9c9-21d51adca52e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.108281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.108905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.109195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.109478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.110177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:15.110488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.111987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.128903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 330/643] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:15.128903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.131473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.147374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.197289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561247c-cad0-4c4a-82c7-36b581e6eca6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.236950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.237565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.237848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.238122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.238833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:15.239135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.239910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840a000-f46f-4f1e-9a6b-90f5cd5ddbdd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.240555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.252956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 314/644] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:15.252956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.284462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.285218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.285573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.286012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.286788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:15.287171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.288731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.306434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 331/645] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:15.307652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.368613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.370138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c4340-7196-4c08-a3af-29f247a21315 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.395036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.409886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.410612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.410960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.411301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.411443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8fa8b-1c6b-4904-9223-e9a8d05836d5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.421934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:15.426946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:15.428212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:15.429399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:15.436844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 315/646] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:15.437688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.454568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.455216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.455600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.455887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.462874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.464674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:15.476722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:15.477609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:15.478525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 332/647] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:15.479021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.546691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9352e5-9add-46a4-b2eb-3e1144d87a05 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.581483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.582258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.582654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.583028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.583853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801a64-4e4d-4578-a9c2-76576fefc2dd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.594952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.596543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.605442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:15.611135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:15.612286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:15.613394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:15.613394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:15.613394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:15.613394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:15.618313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 316/648] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:15.620298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.623226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.623962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.624315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.624654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.655253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.697037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.698528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.726006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642428c-8f39-424c-819e-6afe930deb31 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.766827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.767452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.767764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.768035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.768773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:15.769077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.770436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.779928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 317/649] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:15.780537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.822304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 333/650] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:15.822304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.842209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.887945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:15.892423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725359de-f5f3-4dec-8417-cc17277a8463 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.927100 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924432b-62c6-4746-9648-aa090e50768d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:15.927628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.928260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.928546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.928825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.929621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:15.931048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.933266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.946721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 318/651] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:15.947778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:15.971169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:15.971843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:15.972127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:15.972400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:15.973094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:15.973396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:15.974763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:15.986668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 334/652] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:15.987279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.061545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9558636-2dac-429d-8f69-57d3163f0d43 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.074855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.095903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5823b-ac5b-452d-aa91-ceb4c8b6f245 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.097604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.100676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.100676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.100676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.109937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.111548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:16.128730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:16.129847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:16.130955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:16.130955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:16.130955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:16.130955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:16.130955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 319/653] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:16.130955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.138196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.138829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.139118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.139400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.148970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.151605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.154340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:16.170413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:16.171569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:16.177270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:16.177270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:16.177270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:16.177270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:16.177270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 335/654] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:16.177270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.245440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816cb5f-6e86-427e-912a-107c51bc74f1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.275632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9b03-971d-4ce3-aad2-54957c0b1718 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.279377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.280026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.280313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.280589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.281378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=2f2160adab864eb9970b7767eeeb4789) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:16.281687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.283129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.288841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 320/655] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/role_assignments?user.id=2f2160adab864eb9970b7767eeeb4789&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:16.289613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.312634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.319874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.330974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.330974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.330974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.333029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.334514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:16.348114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:16.349188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 336/656] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:16.356099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.382938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.411006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c091924-0d3c-4605-b4c0-87ed7fd5f182 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.453946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a5d38-5936-40d5-bb21-74cb9eebdf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.454511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5d38-5936-40d5-bb21-74cb9eebdf09 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.454797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5d38-5936-40d5-bb21-74cb9eebdf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.458501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5d38-5936-40d5-bb21-74cb9eebdf09 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.458501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a5d38-5936-40d5-bb21-74cb9eebdf09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:16.458501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a5d38-5936-40d5-bb21-74cb9eebdf09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.458501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a5d38-5936-40d5-bb21-74cb9eebdf09 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.470221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c59733-238b-4123-9f05-8ae863f36b2c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.472175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 321/657] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:16.473540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b662-4d67-4a28-bc0f-6a7b5499a176 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.473826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b662-4d67-4a28-bc0f-6a7b5499a176 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.474098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b662-4d67-4a28-bc0f-6a7b5499a176 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.475302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 322/658] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:16.476049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87b662-4d67-4a28-bc0f-6a7b5499a176 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.512520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf262ed-9ecf-4f61-9829-d8e321aab2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.513171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf262ed-9ecf-4f61-9829-d8e321aab2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.513461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf262ed-9ecf-4f61-9829-d8e321aab2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.513745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf262ed-9ecf-4f61-9829-d8e321aab2b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.514447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf262ed-9ecf-4f61-9829-d8e321aab2b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:16.514768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf262ed-9ecf-4f61-9829-d8e321aab2b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.516328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf262ed-9ecf-4f61-9829-d8e321aab2b6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.531122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 337/659] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:16.532875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3200a7-b810-4b4a-9ee5-e1754c9fbc95 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.533175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3200a7-b810-4b4a-9ee5-e1754c9fbc95 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.533458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3200a7-b810-4b4a-9ee5-e1754c9fbc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.552652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1f3200a7-b810-4b4a-9ee5-e1754c9fbc95 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:16.553867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1f3200a7-b810-4b4a-9ee5-e1754c9fbc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:16.585256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87b662-4d67-4a28-bc0f-6a7b5499a176 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.604930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.620547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.621711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.621999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.622275 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.623074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:16.623384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.624890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.626655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.628164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 338/660] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:16.629201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3200a7-b810-4b4a-9ee5-e1754c9fbc95 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.649312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 323/661] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:16.650470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.745240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3200a7-b810-4b4a-9ee5-e1754c9fbc95 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.758214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd8837-c674-430a-92ad-d4ec11b36ca8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.786748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.787404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.787724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.787998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.795596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.796187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.796598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.797020 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.797494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.797699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.805018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:16.806641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:16.810956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:16.812103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 339/662] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:16.813043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:16.822257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:16.823394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:16.825134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:16.826899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 324/663] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:16.827747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:16.872851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.886969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:16.934686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a95524-225c-4c93-ae9b-ba253a122909 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.938753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55816c-3ffa-407c-8ca7-e2a57989dd43 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:16.972384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.973035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.973325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.973609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.978859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:16.979833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:16.980655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:16.981121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:16.984661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:16.990047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:16.991028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 325/664] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:16.992646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.060492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.062025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.099032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077ec89-ac20-4868-a1ba-858a75808019 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.110027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.114288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.135062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.135754 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.136043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.136322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.137664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:17.137981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.139422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.159348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 326/665] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:17.160485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.179335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 340/666] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:17.180213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.274595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1907ea8-fd70-413c-bbd2-77e6d22f4e70 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.286692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f257d7-f728-436e-8699-60f9a05b6498 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.309230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.309847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.310134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.310400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.311204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:17.311456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.312906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.324641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 327/667] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:17.325453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.335258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.335912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.336195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.336472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.337188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:17.337490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.339121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.349076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 341/668] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:17.349788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.353890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.362764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.439743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909f201-962c-4549-9fe9-bb09ee9d9cce None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.455117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105cc3-1f65-47f5-b2d3-6ed405fb7e3b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.476842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.477715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.477828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.478047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.492949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.493591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.493874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.494148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.505994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.507475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:17.525053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:17.526172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:17.527828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:17.527828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:17.527828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:17.527828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:17.527828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 342/669] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:17.528112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.556731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.558943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.593683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.673289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 328/670] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:17.673289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.709102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.758290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6853b-c924-4a84-8dc3-773948ecaf56 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.782820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cfa79-2177-4de7-bf2c-55c9bffbfbba None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.795727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.796360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.796647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.796943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.797743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=659b367f1ce643a68d55ff1c00d8b7b8) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:17.798060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.799553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.806113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 343/671] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/role_assignments?user.id=659b367f1ce643a68d55ff1c00d8b7b8&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:17.806717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.821276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.830330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.831391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.832698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.840142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.851053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.852662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:17.877732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:17.878913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:17.880064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:17.880064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:17.880064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:17.880064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:17.886162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 329/672] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:17.886162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:17.938637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db0da6-d58a-4062-800c-5a416b0d8cdc None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:17.962815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:17.985422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:17.986574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:17.986868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:17.987147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:17.988113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:17.988519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:17.990075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.005931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 344/673] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:18.007102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.032406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3d7c2-c437-492b-b09b-7f4dd12c5de4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.080474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.081138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.081419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.081691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.095920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.097582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:18.116075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:18.117018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:18.117961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:18.117961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:18.117961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:18.117961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:18.117961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 330/674] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:18.118827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.141121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875114f-5089-4525-b694-2f34690013f8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.170656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:18.178553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.187867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.187867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.187867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.187867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:18.187867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.187867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.201020 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 345/675] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:18.202418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.248644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7e2d1-f10b-4521-9912-1454088857d8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.259890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:18.292774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.293793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.294494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.294980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.305033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.307252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:18.323302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:18.324973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:18.325898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:18.328620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 331/676] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:18.329576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.338174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1b611-2e67-4984-9284-18f1e10925c3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.379339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.380067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.380451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.380829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.382147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:18.382588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.384623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.403682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 346/677] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:18.404794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.450253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:18.458926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f41f-8475-49dc-a9c3-79d83e1092cb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.499509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.500724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.501270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.501546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.510933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.512695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.522795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:18.524733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:18.525703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:18.526678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 332/678] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:18.527345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.530814 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791e28-c2e0-478d-ac7e-a18cd618d620 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.568239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41979-e609-4252-b29f-3febccdb83dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.569211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41979-e609-4252-b29f-3febccdb83dc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.569211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41979-e609-4252-b29f-3febccdb83dc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.569425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41979-e609-4252-b29f-3febccdb83dc None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.570130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41979-e609-4252-b29f-3febccdb83dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:18.570438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41979-e609-4252-b29f-3febccdb83dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.571905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41979-e609-4252-b29f-3febccdb83dc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.584379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 347/679] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:18.585646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee589e3-54b7-4125-a8d3-a9a5c2cff106 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.585933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee589e3-54b7-4125-a8d3-a9a5c2cff106 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.586207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee589e3-54b7-4125-a8d3-a9a5c2cff106 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.587517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 348/680] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:18.592624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee589e3-54b7-4125-a8d3-a9a5c2cff106 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.640453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0848db-0a22-463b-99ed-2cd1c105a35d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.676226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.676882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.677174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.677452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.684944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.686393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:18.697855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:18.698762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 333/681] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:18.699857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.714046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee589e3-54b7-4125-a8d3-a9a5c2cff106 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.721199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:18.750066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257a1b7-c3b7-4118-a3fc-90a2a12b80c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.750688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257a1b7-c3b7-4118-a3fc-90a2a12b80c8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.750972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257a1b7-c3b7-4118-a3fc-90a2a12b80c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.751250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257a1b7-c3b7-4118-a3fc-90a2a12b80c8 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.754673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257a1b7-c3b7-4118-a3fc-90a2a12b80c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:18.754673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257a1b7-c3b7-4118-a3fc-90a2a12b80c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.754673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257a1b7-c3b7-4118-a3fc-90a2a12b80c8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.781871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 349/682] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:18.781871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09155d9-cb9a-465f-9cbd-1a1d38bc7534 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.781871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09155d9-cb9a-465f-9cbd-1a1d38bc7534 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.781871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09155d9-cb9a-465f-9cbd-1a1d38bc7534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.799652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d09155d9-cb9a-465f-9cbd-1a1d38bc7534 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:18.800970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d09155d9-cb9a-465f-9cbd-1a1d38bc7534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:18.815482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:18.819198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89336e5-c25b-4f34-87e0-dfb66226bba9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:18.854778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:18.855642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:18.856002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:18.856348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:18.857210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=ad3cf8a3e0a346fa867445d6b6300447) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:18.857591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:18.859091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:18.865166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 334/683] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/role_assignments?user.id=ad3cf8a3e0a346fa867445d6b6300447&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:18.866042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.882647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 350/684] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:18.884013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09155d9-cb9a-465f-9cbd-1a1d38bc7534 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:18.987469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:19.015649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3919e-c7b7-4cdd-b187-0e12fa81a352 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.052584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09155d9-cb9a-465f-9cbd-1a1d38bc7534 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.064273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.065211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.065522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.065910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.066673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:19.066982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.068444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:19.084125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 335/685] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:19.085316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:19.099585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.100228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.100514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.100794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.122611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:19.190123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.192755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:19.203251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcda72-03b6-4235-b3e2-28c9135f8d60 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.240155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.241259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.241259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.241659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.249817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:19.254454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.256175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:19.272796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:19.273928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:19.275016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:19.275016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:19.275016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:19.275016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:19.280844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 336/686] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:19.280844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:19.341410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 351/687] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:19.341410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:19.392930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:19.415645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f055e-141f-4bb0-ab60-615f3579cbae None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.446740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702285b0-e3f6-41c3-bfde-ef70fddcb0d3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.461189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.463202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.463987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.465001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.466266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=317df4b0baa54808af9b7918ff5c3515) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:19.467136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.468794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:19.477431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 337/688] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:19 2022] GET /identity/v3/role_assignments?user.id=317df4b0baa54808af9b7918ff5c3515&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:19.478471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:19.529652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.530480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.530609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.530817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.531559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:19.531974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.533321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:19.547919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:19.565578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 352/689] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:19.571382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:19.602215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b32f-48cf-4adb-bff0-0df8db90c668 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.645497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.646429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.646711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.646984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.663596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.665191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:19.684336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:19.685658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:19.686753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:19.686753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:19.686753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:19.686753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:19.686753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 338/690] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:19.691962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:19.777874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2bade-1e9a-4d83-bf4b-0381caea06b5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.793442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:19.825588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9c488-7ff1-4ea8-88fc-e09b1f5f59d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.826505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9c488-7ff1-4ea8-88fc-e09b1f5f59d7 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.826995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9c488-7ff1-4ea8-88fc-e09b1f5f59d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.827440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9c488-7ff1-4ea8-88fc-e09b1f5f59d7 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.828380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9c488-7ff1-4ea8-88fc-e09b1f5f59d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.830012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9c488-7ff1-4ea8-88fc-e09b1f5f59d7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:19.948408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b83e-39eb-4c78-a45d-6c2e2892e640 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:19.956675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:19.985796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5889891-7828-4407-8dc0-70af577dbc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:19.986420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889891-7828-4407-8dc0-70af577dbc50 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:19.986702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889891-7828-4407-8dc0-70af577dbc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:19.986980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889891-7828-4407-8dc0-70af577dbc50 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:19.987706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889891-7828-4407-8dc0-70af577dbc50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:19.988018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889891-7828-4407-8dc0-70af577dbc50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:19.989388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5889891-7828-4407-8dc0-70af577dbc50 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:20.001774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 339/691] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:20.004812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22f2bd-3138-4f04-9457-1825dd629e94 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.004812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22f2bd-3138-4f04-9457-1825dd629e94 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.004812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22f2bd-3138-4f04-9457-1825dd629e94 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.005097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 340/692] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:20.007651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22f2bd-3138-4f04-9457-1825dd629e94 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:20.109299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22f2bd-3138-4f04-9457-1825dd629e94 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:20.143946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 353/693] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:42:19 2022] POST /identity/v3/users => generated 254 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:20.145851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef752494-c50f-4ee3-b253-a43ef5085d5f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.146356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef752494-c50f-4ee3-b253-a43ef5085d5f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.146924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef752494-c50f-4ee3-b253-a43ef5085d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.157513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:20.158190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.158559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.158845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.171066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ef752494-c50f-4ee3-b253-a43ef5085d5f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:20.172799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:20.173111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ef752494-c50f-4ee3-b253-a43ef5085d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:20.174352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:20.198190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:20.199944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:20.200859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:20.202054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 341/694] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:20.206690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:20.269000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:20.303652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 354/695] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:20.354372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef752494-c50f-4ee3-b253-a43ef5085d5f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:20.378422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713338b-cccf-4a4b-9ada-150b9378fbc3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:20.415724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:20.416752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.417312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.418443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.419408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:20.419960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:20.421816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:20.435618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 342/696] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:20.440877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:20.460891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:20.468145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef752494-c50f-4ee3-b253-a43ef5085d5f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:20.509487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:20.510878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.511996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.512374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.536305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:20.541693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:20.543485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:20.544375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:20.547073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 355/697] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:20.554367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:20.557113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:20.577900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c72f7-8412-4e41-a905-330acb945694 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:20.621852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:20.622782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.623135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.623484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.624716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=262558e0503a4f57a619c75acfdae331) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:20.625232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:20.627109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:20.635594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 343/698] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:20 2022] GET /identity/v3/role_assignments?user.id=262558e0503a4f57a619c75acfdae331&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:20.681515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3381d-61c2-4fa1-98db-45e8b592b71c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:20.722923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:20.723581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.723869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.724150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.724946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:20.725261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:20.726675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:20.737605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 356/699] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:20.743348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:20.856852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87faac2-bf63-463c-9072-4c9ea9d0673b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:20.893369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:20.893955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:20.894333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:20.894591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:20.905870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:20.907335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:20.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:20.923483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:20.924434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:20.924434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:20.924434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:20.924434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:20.924434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 344/700] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:20.929567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:20.950886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:21.039804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93657a-455e-4db3-8c34-6f8adefcf720 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:21.075846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010cebc-c445-4133-a323-4f47588a6494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:21.076471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010cebc-c445-4133-a323-4f47588a6494 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.076757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010cebc-c445-4133-a323-4f47588a6494 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.077034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010cebc-c445-4133-a323-4f47588a6494 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.077732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010cebc-c445-4133-a323-4f47588a6494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:21.078036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010cebc-c445-4133-a323-4f47588a6494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:21.079453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010cebc-c445-4133-a323-4f47588a6494 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:21.092540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 357/701] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:21.098050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:21.101081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:21.200983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6081c8-f0b5-42d7-87be-154b8023f55c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.201357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6081c8-f0b5-42d7-87be-154b8023f55c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.201644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6081c8-f0b5-42d7-87be-154b8023f55c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.203877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 358/702] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:21.209196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f489b-4cb3-43d3-9be6-33ecb6d1c2d1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.215278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f489b-4cb3-43d3-9be6-33ecb6d1c2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.215278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f489b-4cb3-43d3-9be6-33ecb6d1c2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.215659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b71504-c167-4e8e-95ad-2943f4d29f73 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:21.233728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e49f489b-4cb3-43d3-9be6-33ecb6d1c2d1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:21.234936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e49f489b-4cb3-43d3-9be6-33ecb6d1c2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:21.252303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:21.253064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.253348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.253979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.263382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:21.264986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:21.280028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:21.282292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:21.284137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 345/703] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:42:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:21.291236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:21.328257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 359/704] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:21.356943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f489b-4cb3-43d3-9be6-33ecb6d1c2d1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:21.363918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:21.368213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:21.417167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02750951-d50b-4b5f-8181-51761a1921e6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:21.475481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:21.476652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.477809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.478475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.481145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:21.482274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:21.484814 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:21.521247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 346/705] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:42:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:21.528228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:21.529196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f489b-4cb3-43d3-9be6-33ecb6d1c2d1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:21.597778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:21.598431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.598716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.598994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] PATH_INFO: `/v3/services/network` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.599659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:21.601048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:21.623016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:21.624165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:21.625032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:21.625785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 360/706] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:21 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:21.634843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:21.721704 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1497706-11a6-41d5-8bf3-b3a21cd32823 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:21.742170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:21.767659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:21.786182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c0910-e91f-4657-8f06-34fafdc5e1cf None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:21.806899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a338-365a-4bda-8d2b-9919e96cfb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:21.814016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a338-365a-4bda-8d2b-9919e96cfb07 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.814016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a338-365a-4bda-8d2b-9919e96cfb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.814016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a338-365a-4bda-8d2b-9919e96cfb07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.840694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:21.843123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:21.843584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:21.844448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:21.846096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] RBAC: Adding query filter params (name=network) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:21.847450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:21.849051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:21.860386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 361/707] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:21.865896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:21.924301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95a338-365a-4bda-8d2b-9919e96cfb07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:21.926113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95a338-365a-4bda-8d2b-9919e96cfb07 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.038483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee891a8-594e-4e64-adc8-c2f2d2893163 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.081748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 347/708] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:22.083099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca16e1-2a84-42a0-8ab7-885a7c2afbea None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.083390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca16e1-2a84-42a0-8ab7-885a7c2afbea None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.083706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca16e1-2a84-42a0-8ab7-885a7c2afbea None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.085221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 348/709] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:22.086800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca16e1-2a84-42a0-8ab7-885a7c2afbea None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.087094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2f28f-63c6-4e27-8133-5e43ec506af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.088179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2f28f-63c6-4e27-8133-5e43ec506af5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.088474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2f28f-63c6-4e27-8133-5e43ec506af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.088755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2f28f-63c6-4e27-8133-5e43ec506af5 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.089490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2f28f-63c6-4e27-8133-5e43ec506af5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:22.089827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2f28f-63c6-4e27-8133-5e43ec506af5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.095591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2f28f-63c6-4e27-8133-5e43ec506af5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.097191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:22.102741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 362/710] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:22.104055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703dbb5-15ba-4544-9917-dcca40bcc49b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.104484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703dbb5-15ba-4544-9917-dcca40bcc49b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.105293 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703dbb5-15ba-4544-9917-dcca40bcc49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.132254 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a703dbb5-15ba-4544-9917-dcca40bcc49b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:22.134761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a703dbb5-15ba-4544-9917-dcca40bcc49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:22.137105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:22.287670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 363/711] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:22.295269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca16e1-2a84-42a0-8ab7-885a7c2afbea None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.320749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703dbb5-15ba-4544-9917-dcca40bcc49b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.341878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.342857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.343150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.343425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.357899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.359364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:22.379018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:22.380012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:22.380932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:22.380932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:22.380932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:22.380932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:22.380932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 349/712] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:22.386531 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.434943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703dbb5-15ba-4544-9917-dcca40bcc49b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.463624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:22.464763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:22.473637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.474546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:22.493548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:22.494219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:22.495210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:22.497053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 364/713] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:22.497571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d423393-1200-4df6-ac24-7d5837b4dadd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.499243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.535943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.536565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.536850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.537124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.537826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:22.538133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.539559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.554064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 350/714] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:22.559917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.613546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e5411-36e5-48c2-8e98-210b05b2b549 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.649894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.650924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.651473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.652141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.653079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:22.653598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.655213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.664877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 365/715] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:22.666423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.671052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a829-7863-43bb-842f-4b03c2cf04b1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.704618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:22.710194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.710904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.711195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.711481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.719583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.721281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.727641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:22.736070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:22.737358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:22.738255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:22.740507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 351/716] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:22.747216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.780405 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c620fc5-29ad-4083-9544-f5c2be12206b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.824577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51fedf-02b6-45e0-ac17-3fe29da1c52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.825133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51fedf-02b6-45e0-ac17-3fe29da1c52c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.825419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51fedf-02b6-45e0-ac17-3fe29da1c52c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.829173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51fedf-02b6-45e0-ac17-3fe29da1c52c None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.829173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51fedf-02b6-45e0-ac17-3fe29da1c52c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:22.829173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51fedf-02b6-45e0-ac17-3fe29da1c52c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.829173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51fedf-02b6-45e0-ac17-3fe29da1c52c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.842801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 366/717] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:22.879097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51951ae-eaf2-4627-9d34-5bd1daebb767 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:22.923592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:22.924245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.924525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.924795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.925843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:22.927271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52a69-1dc2-4c0c-8eb2-c6a26178e21f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.927579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52a69-1dc2-4c0c-8eb2-c6a26178e21f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.927844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52a69-1dc2-4c0c-8eb2-c6a26178e21f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.929242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 367/718] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:22.929663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:22.931266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:22.935751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b1311-33a1-4986-bff7-ecd64491645a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:22.936052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b1311-33a1-4986-bff7-ecd64491645a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:22.936323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b1311-33a1-4986-bff7-ecd64491645a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:22.948218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 352/719] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:22.953840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:22.957671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-239b1311-33a1-4986-bff7-ecd64491645a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:22.958901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-239b1311-33a1-4986-bff7-ecd64491645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:23.037831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:23.039422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 368/720] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:42:23.065524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b1311-33a1-4986-bff7-ecd64491645a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:23.087500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8376-fa08-46e1-b90e-3ae08b4cac36 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:23.122910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:23.126324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:23.127127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:23.127608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:23.128028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:23.128957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=659b367f1ce643a68d55ff1c00d8b7b8) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:23.129416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:23.131059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:23.139334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 353/721] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:22 2022] GET /identity/v3/role_assignments?user.id=659b367f1ce643a68d55ff1c00d8b7b8&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:23.182497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b1311-33a1-4986-bff7-ecd64491645a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:23.224457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:23.225308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:23.225739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:23.226206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:23.247346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:23.252723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:23.253913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:23.254955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:23.261984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 369/722] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:23.268642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:23.394936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb4ba4-a554-437b-8789-2c1e4a6b0472 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:23.413859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:23.438139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89465ecb-ef97-496b-8590-c022860da33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:23.439087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89465ecb-ef97-496b-8590-c022860da33b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:23.439558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89465ecb-ef97-496b-8590-c022860da33b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:23.439979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89465ecb-ef97-496b-8590-c022860da33b None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:23.440861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89465ecb-ef97-496b-8590-c022860da33b None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:23.441324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89465ecb-ef97-496b-8590-c022860da33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:23.442861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89465ecb-ef97-496b-8590-c022860da33b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:23.457869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 354/723] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:23.458998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:23.618272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823ca9e-f388-4bff-b3d5-ad54e4563d05 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:23.633650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:23.663649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:23.664238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:23.664526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:23.664804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:23.671328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb11f1-cd91-4d13-8e1b-5e4ca1a31ad2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:23.671640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb11f1-cd91-4d13-8e1b-5e4ca1a31ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:23.671912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb11f1-cd91-4d13-8e1b-5e4ca1a31ad2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:23.673486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 355/724] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:23.679084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:23.682226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:23.682345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57203939-5e1d-4d14-8a6c-9d6adbeaf142 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:23.682345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57203939-5e1d-4d14-8a6c-9d6adbeaf142 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:23.682345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57203939-5e1d-4d14-8a6c-9d6adbeaf142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:23.705867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:23.706867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:23.707805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:23.707805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:23.707805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:23.707805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:23.708943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 370/725] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:23.712447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:23.714744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-57203939-5e1d-4d14-8a6c-9d6adbeaf142 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:23.717849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-57203939-5e1d-4d14-8a6c-9d6adbeaf142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:23.853255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 356/726] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:23.875491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8137fd-191d-4b76-aa88-7fc436cd77b7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:23.887578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57203939-5e1d-4d14-8a6c-9d6adbeaf142 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:23.923524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:23.925308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:23.926283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:23.927622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:23.928481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:23.930241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:23.931805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:23.945692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 371/727] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:23.953426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:24.007886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:24.087368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57203939-5e1d-4d14-8a6c-9d6adbeaf142 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:24.105874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:24.121044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92af8e-a428-4411-b4cf-985e4be7f0b3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:24.154078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:24.154755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:24.155039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:24.155315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:24.176590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:24.180726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:24.181854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:24.183274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 357/728] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:24.193591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:24.200314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:24.200991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:24.201285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:24.201576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:24.209592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:24.211070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:24.224120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:24.225974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:24.227146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 372/729] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:42:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:24.236479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:24.335889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:24.376577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a7250-8668-422f-8b1f-878bf987b1a7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:24.387316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d35f52-62bc-46bf-84ca-78e2e6c30ca9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:24.424155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:24.424774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:24.425052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:24.425324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:24.426039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:24.426344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:24.427787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:24.443746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 358/730] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:24.450110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:24.451819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:24.452447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:24.452727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:24.453000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:24.453800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:24.454106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:24.455570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:24.475720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 373/731] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 07:42:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:24.485061 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:24.515193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:24.649582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a99a-dd9f-4fda-ba58-50501217ad9b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:24.686841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c191-57e1-4ea4-82c8-75f51256f01e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:24.716980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:24.717635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:24.717921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:24.718199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:24.737344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:24.738948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:24.755406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:24.756384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:24.757286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:24.757286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:24.757286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:24.757286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:24.757286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 359/732] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:24.758827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:24.763562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:24.763562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:24.763562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:24.766084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:24.789216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:24.896277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:24.898137 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:24.901889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:24.923562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3be29-9773-4cbc-bfae-8db4bf7bfbd8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:24.988322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:24.989095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:24.989221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:24.989407 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:24.990111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:24.990418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:24.991821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.013164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 360/733] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:25.018584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.027560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 374/734] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:42:25.027942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.075882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.153216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1e4e6-19b7-470f-b524-f0d777a2b481 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:25.153836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c52d97-a288-4e8d-879d-67a362c38639 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:25.198447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84332-dde2-4df1-a7a6-46dc3435ce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:25.199642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84332-dde2-4df1-a7a6-46dc3435ce6b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.200133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84332-dde2-4df1-a7a6-46dc3435ce6b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.200637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84332-dde2-4df1-a7a6-46dc3435ce6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.202968 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c22ab-ed2a-4e86-a00b-64937b522860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:25.203989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c22ab-ed2a-4e86-a00b-64937b522860 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.204281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c22ab-ed2a-4e86-a00b-64937b522860 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.204561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c22ab-ed2a-4e86-a00b-64937b522860 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.211602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84332-dde2-4df1-a7a6-46dc3435ce6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:25.212845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c22ab-ed2a-4e86-a00b-64937b522860 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:25.213333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84332-dde2-4df1-a7a6-46dc3435ce6b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.214773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c22ab-ed2a-4e86-a00b-64937b522860 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-a54c22ab-ed2a-4e86-a00b-64937b522860 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:25.227051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:25.228274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:25.230294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 361/735] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fbe84332-dde2-4df1-a7a6-46dc3435ce6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:25.230951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:25.232470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:25.235031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:25.235031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:25.235031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:25.235031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:25.235605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd78ef-93f5-436f-9ade-8a5ce4f108cc None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.235605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd78ef-93f5-436f-9ade-8a5ce4f108cc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.235605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd78ef-93f5-436f-9ade-8a5ce4f108cc None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.236324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 375/736] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:25.236324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aea95-0e2d-4ed1-a7fe-e060781bc8d9 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.236324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aea95-0e2d-4ed1-a7fe-e060781bc8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.236324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aea95-0e2d-4ed1-a7fe-e060781bc8d9 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.236324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 376/737] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:25.241588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 362/738] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:25.242645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635aea95-0e2d-4ed1-a7fe-e060781bc8d9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.242820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd78ef-93f5-436f-9ade-8a5ce4f108cc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.243439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.341989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.352660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635aea95-0e2d-4ed1-a7fe-e060781bc8d9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:25.355475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd78ef-93f5-436f-9ade-8a5ce4f108cc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:25.389427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768fdc0-239c-466c-9413-7c8c9e20cf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:25.390050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768fdc0-239c-466c-9413-7c8c9e20cf41 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.390335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768fdc0-239c-466c-9413-7c8c9e20cf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.390613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768fdc0-239c-466c-9413-7c8c9e20cf41 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.391418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768fdc0-239c-466c-9413-7c8c9e20cf41 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:25.391518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa5c8b-2491-49dd-9587-0c3380de5013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:25.391783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768fdc0-239c-466c-9413-7c8c9e20cf41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:25.392091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5c8b-2491-49dd-9587-0c3380de5013 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.392377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5c8b-2491-49dd-9587-0c3380de5013 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.392658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5c8b-2491-49dd-9587-0c3380de5013 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.393201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768fdc0-239c-466c-9413-7c8c9e20cf41 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.393353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa5c8b-2491-49dd-9587-0c3380de5013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:25.393658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa5c8b-2491-49dd-9587-0c3380de5013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:25.395150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa5c8b-2491-49dd-9587-0c3380de5013 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.415823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 377/739] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 07:42:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:25.416011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 363/740] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:25.416090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996042-570c-4ea3-85fa-b18775bec093 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.416090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996042-570c-4ea3-85fa-b18775bec093 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.416090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996042-570c-4ea3-85fa-b18775bec093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.416296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cea064-986a-4a94-8d9c-8671c6b87d9d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.416296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cea064-986a-4a94-8d9c-8671c6b87d9d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.416296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cea064-986a-4a94-8d9c-8671c6b87d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.431690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-18996042-570c-4ea3-85fa-b18775bec093 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:25.433564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-18996042-570c-4ea3-85fa-b18775bec093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:25.434109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e4cea064-986a-4a94-8d9c-8671c6b87d9d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:25.435272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e4cea064-986a-4a94-8d9c-8671c6b87d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:25.497557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.504303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 378/741] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:25.505143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996042-570c-4ea3-85fa-b18775bec093 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.527563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 364/742] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:42:25.527563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cea064-986a-4a94-8d9c-8671c6b87d9d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.599175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.618761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996042-570c-4ea3-85fa-b18775bec093 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:25.627897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cea064-986a-4a94-8d9c-8671c6b87d9d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:25.653388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:25.654035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.654321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.654601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.661183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:25.661823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.662102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.662195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:25.662374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.663581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:25.674934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:25.675853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:25.676777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:25.677549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 379/743] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:25.678441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.721037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.730877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:25.732347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.780343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a45f-238d-4e34-91c5-0bee015380cc None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:25.815105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0157f7-c0b2-48c6-8680-a537b47ed55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:25.815755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0157f7-c0b2-48c6-8680-a537b47ed55b None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.816044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0157f7-c0b2-48c6-8680-a537b47ed55b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.816322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0157f7-c0b2-48c6-8680-a537b47ed55b None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.817016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0157f7-c0b2-48c6-8680-a537b47ed55b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:25.818400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0157f7-c0b2-48c6-8680-a537b47ed55b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:25.834563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 365/744] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:42:25.835380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.914388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.920034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 380/745] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:25 2022] POST /identity/v3/services => generated 240 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:25.921322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aada1-095d-41ff-82e6-d72d5ec702a4 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:25.921616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aada1-095d-41ff-82e6-d72d5ec702a4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:25.921900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aada1-095d-41ff-82e6-d72d5ec702a4 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:25.923424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 381/746] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:25.923934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aada1-095d-41ff-82e6-d72d5ec702a4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:25.937641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:25.947612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be2c93-a390-46d6-82e3-8f5702190e38 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.002218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.002860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.003151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.003796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.004194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:26.005624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:26.046607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aada1-095d-41ff-82e6-d72d5ec702a4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.075572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 366/747] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:25 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:26.075572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:26.096485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cb174-130c-4f13-8939-da6cc22a4d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.097136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb174-130c-4f13-8939-da6cc22a4d87 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.097428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb174-130c-4f13-8939-da6cc22a4d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.097708 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb174-130c-4f13-8939-da6cc22a4d87 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.098483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cb174-130c-4f13-8939-da6cc22a4d87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:26.098800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cb174-130c-4f13-8939-da6cc22a4d87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:26.100490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cb174-130c-4f13-8939-da6cc22a4d87 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:26.115466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 382/748] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:26.116935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02cd5d-0548-47e0-b27f-945cb4b02e0f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.117848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02cd5d-0548-47e0-b27f-945cb4b02e0f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.117848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02cd5d-0548-47e0-b27f-945cb4b02e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.141067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8e02cd5d-0548-47e0-b27f-945cb4b02e0f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:26.142440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8e02cd5d-0548-47e0-b27f-945cb4b02e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:26.206371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:26.231560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 383/749] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:26.231560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02cd5d-0548-47e0-b27f-945cb4b02e0f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:26.233612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff05bf-6080-4d4b-abfa-340539507a26 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.279355 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.280210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.280601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.280971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.292822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:26.294593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:26.303929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:26.311437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:26.312409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:26.313592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:26.313592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:26.313592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:26.313592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:26.319164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 367/750] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:26.319164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:26.365931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02cd5d-0548-47e0-b27f-945cb4b02e0f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.420602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.421383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.421791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.422334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.425007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=3001dbc46cc441cfb61b1d63a4461314) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:26.425516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:26.427125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:26.436719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 384/751] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/role_assignments?user.id=3001dbc46cc441cfb61b1d63a4461314&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:26.437752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:26.455108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae07bdf-1be0-440d-b3a1-f3a0689534fd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.504383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:26.544319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.544994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.545277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.545594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.567205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:26.572036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a955ad-d40d-4f87-b476-22b4f59888df None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:26.581135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:26.582398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:26.583453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 368/752] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:26.588550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:26.617915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.618795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.619252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.619723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.621259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:26.621832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:26.623393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:26.636627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 385/753] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:26.642208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:26.691943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:26.749094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19db3-054e-4f61-b6cf-49d3b88b4658 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.772758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:26.790842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94978b5c-752e-4dac-b584-927ec547c4c4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:26.813807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.814697 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.815009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.815296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.831068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:26.832548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:26.853282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:26.854253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:26.855785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:26.860365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 386/754] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:26.866176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:26.866797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:26.867076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:26.867486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:26.868507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:26.868821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:26.870418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:26.876532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:26.880866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 369/755] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:26.886165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:27.071255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7834b-1402-42a3-83ac-2ae1d750c213 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:27.083515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360d8c3-8c04-42bc-af82-a470ce385970 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:27.119841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:27.138801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:27.139868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:27.141758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:27.142107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:27.153936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:27.154659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:27.155014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:27.155368 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:27.156263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:27.156642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:27.158382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:27.166434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:27.168693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:27.184820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 387/756] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:27.190697 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:27.200718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:27.201924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:27.203064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:27.203064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:27.203064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:27.203064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:27.207902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 370/757] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:27.212825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:27.262871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:27.348165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9021eb-bda3-4a02-9772-ae336fea3e87 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:27.385185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:27.386137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:27.386519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:27.386899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:27.387977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:27.388424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:27.390308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:27.404005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 371/758] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:27.409235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:27.441202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bf06d-f57b-4d52-8aeb-4f0a04eafc9a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:27.489800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:27.490424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:27.490706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:27.490980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:27.491796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=262558e0503a4f57a619c75acfdae331) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:27.492105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:27.496008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:27.505734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 388/759] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:27 2022] GET /identity/v3/role_assignments?user.id=262558e0503a4f57a619c75acfdae331&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:27.508345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:27.615718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192535b-e1d6-4962-a317-defe03b7e5c5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:27.656819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:27.657369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:27.657652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:27.657922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:27.666429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:27.668891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:27.683068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:27.684246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:27.685221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:27.687961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 372/760] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:42:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:27.695569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:27.849335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:27.859349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9874-e14d-4133-a88a-95df93b78b48 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:27.903127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:27.903760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:27.904048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:27.904377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:27.905200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:27.905511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:27.906969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:27.931341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 389/761] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:42:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:27.937252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.090962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8a8d-aa67-4cf5-b656-7319139df037 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:28.105190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:28.139200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac16055-5ff0-41ee-beda-5ca0d24cc642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:28.139889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16055-5ff0-41ee-beda-5ca0d24cc642 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.140223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16055-5ff0-41ee-beda-5ca0d24cc642 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.140528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16055-5ff0-41ee-beda-5ca0d24cc642 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.152173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac16055-5ff0-41ee-beda-5ca0d24cc642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:28.159441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac16055-5ff0-41ee-beda-5ca0d24cc642 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3ac16055-5ff0-41ee-beda-5ca0d24cc642 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:28.175231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:28.184091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:28.185023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:28.185023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:28.185023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:28.185023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:28.185023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 373/762] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:28.186827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.323905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5c7c-aa28-4e5c-ae88-5ad01ddf02a0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:28.376257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:28.376891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.377180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.377458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.378167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:28.378475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:28.379884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:28.393966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef034d-87f2-4625-9eba-ac8acd92c57e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.394321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef034d-87f2-4625-9eba-ac8acd92c57e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.394609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef034d-87f2-4625-9eba-ac8acd92c57e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.396266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 374/763] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:28.397009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 390/764] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:28.401308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa772ce-8fa5-4cd3-8bf0-781d6cef728d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.402067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa772ce-8fa5-4cd3-8bf0-781d6cef728d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.402360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa772ce-8fa5-4cd3-8bf0-781d6cef728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.402980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.421568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-caa772ce-8fa5-4cd3-8bf0-781d6cef728d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:28.422763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-caa772ce-8fa5-4cd3-8bf0-781d6cef728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:28.427859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:28.523025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 375/765] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:28.536773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d4e76-ade0-4d31-ad97-0446ba907a95 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:28.546627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa772ce-8fa5-4cd3-8bf0-781d6cef728d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.573508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2be7da-8395-4951-bada-3756b4fea3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:28.574342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2be7da-8395-4951-bada-3756b4fea3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.574731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2be7da-8395-4951-bada-3756b4fea3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.575017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2be7da-8395-4951-bada-3756b4fea3b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.582653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2be7da-8395-4951-bada-3756b4fea3b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:28.584315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2be7da-8395-4951-bada-3756b4fea3b6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ec2be7da-8395-4951-bada-3756b4fea3b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:28.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:28.598416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:28.599325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 391/766] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:28.600326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958bcfb-27b4-47e8-ab8c-20cd8ffefebe None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.602101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958bcfb-27b4-47e8-ab8c-20cd8ffefebe None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.602101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958bcfb-27b4-47e8-ab8c-20cd8ffefebe None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.602429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 392/767] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:28.604897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958bcfb-27b4-47e8-ab8c-20cd8ffefebe None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.628655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:28.653345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa772ce-8fa5-4cd3-8bf0-781d6cef728d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:28.683422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:28.694276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c969a-b99d-4d82-9712-41856c9342e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:28.695312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c969a-b99d-4d82-9712-41856c9342e9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.695705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c969a-b99d-4d82-9712-41856c9342e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.698458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c969a-b99d-4d82-9712-41856c9342e9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.698458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c969a-b99d-4d82-9712-41856c9342e9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:28.699443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c969a-b99d-4d82-9712-41856c9342e9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b50c969a-b99d-4d82-9712-41856c9342e9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:28.714862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:28.716007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:28.716853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:28.719612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 376/768] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:28 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:28.721126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ee69-3658-4652-a6a3-40e2d94dc832 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.721617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ee69-3658-4652-a6a3-40e2d94dc832 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.722091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ee69-3658-4652-a6a3-40e2d94dc832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.727699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958bcfb-27b4-47e8-ab8c-20cd8ffefebe None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:28.745375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5556ee69-3658-4652-a6a3-40e2d94dc832 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:28.746713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5556ee69-3658-4652-a6a3-40e2d94dc832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:28.781435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:28.782482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.783097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.783714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.784735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:28.785249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:28.786954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:28.800211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 393/769] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:28.801441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.822045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 377/770] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:28.823568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ee69-3658-4652-a6a3-40e2d94dc832 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.903413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:28.910006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ec98-ee07-4182-b354-e212e63f882c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:28.937604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ee69-3658-4652-a6a3-40e2d94dc832 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:28.942622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:28.947811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:28.948552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.948863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.949149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.949838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] RBAC: Adding query filter params (name=network) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:28.950149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:28.951558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:28.961314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 394/771] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:28.962045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:28.978702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:28.980250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:28.980699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:28.980980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/659b367f1ce643a68d55ff1c00d8b7b8/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:28.983007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=659b367f1ce643a68d55ff1c00d8b7b8, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:28.984498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.064291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583f5ff-d5a3-4891-b967-128d8864540f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.091352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 378/772] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:28 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/659b367f1ce643a68d55ff1c00d8b7b8/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:29.092877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.113067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf9748-da7c-4151-9cc2-6cec42886da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.113749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf9748-da7c-4151-9cc2-6cec42886da4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.114038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf9748-da7c-4151-9cc2-6cec42886da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.114373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf9748-da7c-4151-9cc2-6cec42886da4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.115084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf9748-da7c-4151-9cc2-6cec42886da4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.116650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf9748-da7c-4151-9cc2-6cec42886da4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-dbaf9748-da7c-4151-9cc2-6cec42886da4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:29.127666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:29.128696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:29.136515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 395/773] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 07:42:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:29.137397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f728b-3f8e-4438-8a10-9fa5e06858e1 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.137397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f728b-3f8e-4438-8a10-9fa5e06858e1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.137397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f728b-3f8e-4438-8a10-9fa5e06858e1 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.137397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 396/774] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:29.137397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f728b-3f8e-4438-8a10-9fa5e06858e1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.207958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:29.256291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e899fe-48f9-460d-b854-3df041b79063 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.261138 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f728b-3f8e-4438-8a10-9fa5e06858e1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.301104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:29.309404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.310145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.316770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.316770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.316961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea1dce-9233-4e7f-9615-94d566c6629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.317063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] RBAC: Adding query filter params (type=network) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:29.317063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.317227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea1dce-9233-4e7f-9615-94d566c6629d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.317227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea1dce-9233-4e7f-9615-94d566c6629d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.317227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea1dce-9233-4e7f-9615-94d566c6629d None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.317227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea1dce-9233-4e7f-9615-94d566c6629d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:29.317482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.317561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea1dce-9233-4e7f-9615-94d566c6629d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.317561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea1dce-9233-4e7f-9615-94d566c6629d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.320090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 397/775] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:29.320990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 379/776] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:29.321286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702a56a-0c78-4c05-b62b-2e0a54511f06 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.321574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702a56a-0c78-4c05-b62b-2e0a54511f06 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.321853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702a56a-0c78-4c05-b62b-2e0a54511f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.326997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.341363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f702a56a-0c78-4c05-b62b-2e0a54511f06 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:29.342558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f702a56a-0c78-4c05-b62b-2e0a54511f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:29.419999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 398/777] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:29.420935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702a56a-0c78-4c05-b62b-2e0a54511f06 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.453425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619ff4e-38d8-48c1-acf3-9f9e6b72420a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.479159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:29.491323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.498354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.498354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.498354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.498354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:29.498354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.498354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.504152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 380/778] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:29.505247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.529167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702a56a-0c78-4c05-b62b-2e0a54511f06 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.565012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.565685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.565983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.566270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.567084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] RBAC: Adding query filter params (interface=public, service_id=0359c6c15ceb4721b565ed5c7b269ea0, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:29.567405 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.568882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.579139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:29.581932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 399/779] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/endpoints?service_id=0359c6c15ceb4721b565ed5c7b269ea0&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:29.582925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.626990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd93d17-df04-47ec-9e40-f1110ef8ab39 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.668231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.668885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.669178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.669459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.680498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:29.686423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:29.687613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 381/780] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:29.688732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.709243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca534621-96e7-43d3-9f0d-209e0df0054d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.734290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:29.756328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.757289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.757720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.758144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.761593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] RBAC: Adding query filter params (interface=public, service_id=0c839e33c0c849cbb02f8d2f15ae9b6a, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:29.761927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.763369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.771354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 400/781] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/endpoints?service_id=0c839e33c0c849cbb02f8d2f15ae9b6a&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:29.772241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.816771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fd1e0-c2df-4956-b14a-ea7011eb3447 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.834924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:29.853941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.855514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.855983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.862386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.862386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.862386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.863618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 382/782] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:29.871809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.896516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbba6f-b641-4d42-91ed-9fc88d7266cb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:29.940340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcbe8e-602b-409d-b640-9d3dc392e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:29.941109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcbe8e-602b-409d-b640-9d3dc392e5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.941460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcbe8e-602b-409d-b640-9d3dc392e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.941806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcbe8e-602b-409d-b640-9d3dc392e5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.942669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcbe8e-602b-409d-b640-9d3dc392e5d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:29.943050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcbe8e-602b-409d-b640-9d3dc392e5d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:29.944586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcbe8e-602b-409d-b640-9d3dc392e5d0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:29.959042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 401/783] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:29.960736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9b71f-0edd-4b1a-9e33-c4a7c9edb2f8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:29.961099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9b71f-0edd-4b1a-9e33-c4a7c9edb2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:29.961610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9b71f-0edd-4b1a-9e33-c4a7c9edb2f8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:29.964171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 402/784] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:29.968509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9b71f-0edd-4b1a-9e33-c4a7c9edb2f8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:29.997983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b840ed5-ebd3-4c2d-99f7-0c2fb881ce2d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.023353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:30.043975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af4f72-ae2b-4af6-8e45-e81dc066ab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.044601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4f72-ae2b-4af6-8e45-e81dc066ab89 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.048811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4f72-ae2b-4af6-8e45-e81dc066ab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.048811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4f72-ae2b-4af6-8e45-e81dc066ab89 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.048811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af4f72-ae2b-4af6-8e45-e81dc066ab89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:30.048811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af4f72-ae2b-4af6-8e45-e81dc066ab89 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:30.055584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 383/785] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:30.055584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5849252-c93f-4f02-a1e9-fee67656239d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.055830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5849252-c93f-4f02-a1e9-fee67656239d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.059560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5849252-c93f-4f02-a1e9-fee67656239d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.076431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f5849252-c93f-4f02-a1e9-fee67656239d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:30.077675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f5849252-c93f-4f02-a1e9-fee67656239d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:30.094502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9b71f-0edd-4b1a-9e33-c4a7c9edb2f8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.129219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:30.136275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.137233 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.137694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.138124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.150568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:30.152939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:30.158084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 384/786] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:30.171063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:30.182314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:30.183914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:30.183914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:30.183914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:30.183914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:30.183914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 403/787] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:30.184260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5849252-c93f-4f02-a1e9-fee67656239d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:30.189517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:30.342995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:30.349673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507ca52-7efa-402b-b576-1471578374f3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.356226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5849252-c93f-4f02-a1e9-fee67656239d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.390466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.391177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.391469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.391819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.405960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.406577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.406859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.407135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.407869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:30.408179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:30.409557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:30.414946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:30.425994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:30.426976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:30.427884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:30.431317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 404/788] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:30.435223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 385/789] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:30.440347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:30.442018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:30.518700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:30.596205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b8b1d-3082-47c2-8a47-5144645eaa1a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.639196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f7cd8-582c-4333-8abc-97d4e6d50f22 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.640272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.640925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.641214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.641495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.642892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:30.644106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:30.645934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:30.665868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 386/790] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:30.671949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:30.719994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.720651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.720937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.721216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.740235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:30.741720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:30.765188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:30.766089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:30.767376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 405/791] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:30.773026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:30.813892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:30.817265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:30.818353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60607ce8-0ecf-40ce-b1c9-8092fa0df0b0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.856070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.856642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.856926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.857201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.869191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:30.874700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:30.886813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:30.888979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:30.891139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:30.891139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:30.891139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:30.891139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:30.891139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 387/792] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:30.893972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.003873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af7661-dfb7-45a9-ae29-5d0899354382 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.033702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4eea-5dff-4e88-886a-0143102e4f82 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.040580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.041212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.041489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.042292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.042998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:31.043301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.044779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.058575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 388/793] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:31.063975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.077130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:31.109189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.109823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.110107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.110379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.111177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:31.111483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.112948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.146323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 406/794] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:31.152956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.180416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27195719-3515-4860-bdf5-6581480d6661 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.220810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.221562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.221853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.222133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.231916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.233492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:31.247277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:31.248303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:31.249279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 389/795] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:31.256417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.326353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:31.336473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:31.352405 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0161-a299-45af-821d-9a8fb81f53fa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.371965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196e24d-ca36-4488-976a-e6c41f5c9f70 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.403647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a11c7-2f01-4c8c-944f-2d17b5e30483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.404300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a11c7-2f01-4c8c-944f-2d17b5e30483 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.404581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a11c7-2f01-4c8c-944f-2d17b5e30483 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.404855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a11c7-2f01-4c8c-944f-2d17b5e30483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.413212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f56200-4e52-4f6b-96e1-7c44f841f2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.413841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f56200-4e52-4f6b-96e1-7c44f841f2ed None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.414130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f56200-4e52-4f6b-96e1-7c44f841f2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.414412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f56200-4e52-4f6b-96e1-7c44f841f2ed None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.415213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f56200-4e52-4f6b-96e1-7c44f841f2ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:31.415543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f56200-4e52-4f6b-96e1-7c44f841f2ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.415649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a11c7-2f01-4c8c-944f-2d17b5e30483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.417056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f56200-4e52-4f6b-96e1-7c44f841f2ed None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.417276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a11c7-2f01-4c8c-944f-2d17b5e30483 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4d0a11c7-2f01-4c8c-944f-2d17b5e30483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:31.433356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:31.434266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:31.435621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:31.435621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:31.435621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:31.435621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:31.435621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 407/796] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:31.436476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b893d-d74c-463a-9552-f1100328c2f7 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.436476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b893d-d74c-463a-9552-f1100328c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.436775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b893d-d74c-463a-9552-f1100328c2f7 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.438162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 408/797] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:31.441117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 390/798] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:31.441451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b893d-d74c-463a-9552-f1100328c2f7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.443586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383b4a0-6e1e-4f72-bf05-9c04a9d28e44 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.443890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383b4a0-6e1e-4f72-bf05-9c04a9d28e44 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.444171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383b4a0-6e1e-4f72-bf05-9c04a9d28e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.467132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c383b4a0-6e1e-4f72-bf05-9c04a9d28e44 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:31.468404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c383b4a0-6e1e-4f72-bf05-9c04a9d28e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:31.545806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 391/799] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:31.546711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383b4a0-6e1e-4f72-bf05-9c04a9d28e44 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.549530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b893d-d74c-463a-9552-f1100328c2f7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.582704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:31.593346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.593977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.594261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.594567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.595268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:31.595594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.602421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.610211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 409/800] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:31.611067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.616587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:31.665381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383b4a0-6e1e-4f72-bf05-9c04a9d28e44 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.703371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.704076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.704387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.704802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.716508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.718430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.721668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69260aa1-293a-41fa-b6dc-4d6d9b2f47ed None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:31.734972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:31.736122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:31.737225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:31.737225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:31.737225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:31.737225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:31.740699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 392/801] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:31.741550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.762678 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.763609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.764036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.764533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.776878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:31.782209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:31.783332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:31.784201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:31.784866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 410/802] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:31.790001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.843866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:31.850561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a080567-05eb-422a-ac9f-af88191531b7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.861965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:31.888615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.889272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.889613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.889842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.898064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.899829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.904171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd209157-a4df-47dc-a6f8-f92bfd4fc3da None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:31.912417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:31.913336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:31.914229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:31.919153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 393/803] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:31.920127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:31.946374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:31.947350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:31.947921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:31.948416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:31.949410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:31.949969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.951628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.964409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 411/804] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:31.965538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.037776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a20ab-c624-4112-bf06-a97efcb667b3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.075202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57af35-9e2a-4d5e-828f-0e63594a4ec2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.076246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.076880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.077170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.077451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.078290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:32.078604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.080327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.084648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:32.091731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 394/805] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:32.092934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.118884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.119508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.119889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.120121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.120861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:32.121175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.122732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.130888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:32.142426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 412/806] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:32.143642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.208555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392b555-876a-4c77-9ad4-fefde988f4ca None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.247140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.247960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.248245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.248837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.254058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaa57a-87fd-42e2-bcd1-2e1cf520d46c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.257703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.259149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:32.274851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:32.276337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 395/807] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:32.282662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.288342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899deafe-4dda-46da-b77f-0d154a4b6b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.288988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899deafe-4dda-46da-b77f-0d154a4b6b11 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.289271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899deafe-4dda-46da-b77f-0d154a4b6b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.289546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899deafe-4dda-46da-b77f-0d154a4b6b11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.301865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899deafe-4dda-46da-b77f-0d154a4b6b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.303458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899deafe-4dda-46da-b77f-0d154a4b6b11 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-899deafe-4dda-46da-b77f-0d154a4b6b11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:32.318402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:32.319331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:32.320805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:32.320805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:32.320805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:32.320805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:32.320805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 413/808] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:32.320805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69533-558d-4a4d-b6e2-6beb34175eb5 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.321180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69533-558d-4a4d-b6e2-6beb34175eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.321257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69533-558d-4a4d-b6e2-6beb34175eb5 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.327734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 414/809] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:32.327734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69533-558d-4a4d-b6e2-6beb34175eb5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.345806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:32.388211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:32.397534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da45a-8404-4614-88ee-4451b58df4ab None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.433802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c1b82-65da-4c4c-8bef-f732acde723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.434482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c1b82-65da-4c4c-8bef-f732acde723e None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.434765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c1b82-65da-4c4c-8bef-f732acde723e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.435039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c1b82-65da-4c4c-8bef-f732acde723e None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/3001dbc46cc441cfb61b1d63a4461314/roles/49afe9a7e8ab49dca85e0aa074a1ee33` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.435474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69533-558d-4a4d-b6e2-6beb34175eb5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.450250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c1b82-65da-4c4c-8bef-f732acde723e None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=3001dbc46cc441cfb61b1d63a4461314, role_id=49afe9a7e8ab49dca85e0aa074a1ee33)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.451737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c1b82-65da-4c4c-8bef-f732acde723e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.471272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.471866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.472152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.472427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.473173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:32.473523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.474962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.487621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 415/810] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:32.490007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.508829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 396/811] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:32 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/3001dbc46cc441cfb61b1d63a4461314/roles/49afe9a7e8ab49dca85e0aa074a1ee33 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:32.510769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e17a-aedc-4408-8fd9-d0f013ce8958 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.511169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e17a-aedc-4408-8fd9-d0f013ce8958 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.511451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e17a-aedc-4408-8fd9-d0f013ce8958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.534562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8f54e17a-aedc-4408-8fd9-d0f013ce8958 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:32.535847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8f54e17a-aedc-4408-8fd9-d0f013ce8958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:32.581773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:32.608454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dbf1e-dc2a-4a38-9717-d75477222e21 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.622154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 397/812] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:32.622154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e17a-aedc-4408-8fd9-d0f013ce8958 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.658456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a2144-1d64-43ca-b79b-5c1bf3998d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.659206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a2144-1d64-43ca-b79b-5c1bf3998d3e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.659592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a2144-1d64-43ca-b79b-5c1bf3998d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.659884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a2144-1d64-43ca-b79b-5c1bf3998d3e None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.660599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a2144-1d64-43ca-b79b-5c1bf3998d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:32.660917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a2144-1d64-43ca-b79b-5c1bf3998d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.662422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a2144-1d64-43ca-b79b-5c1bf3998d3e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.679095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 416/813] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:32.680465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8042f47-114a-423e-885b-053d04341c12 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.680762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8042f47-114a-423e-885b-053d04341c12 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.681039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8042f47-114a-423e-885b-053d04341c12 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.682418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 417/814] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:32.682963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8042f47-114a-423e-885b-053d04341c12 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.708158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:32.741006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e17a-aedc-4408-8fd9-d0f013ce8958 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.782304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.782988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.783274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.783578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/262558e0503a4f57a619c75acfdae331/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.786863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=262558e0503a4f57a619c75acfdae331, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.788605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.803705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8042f47-114a-423e-885b-053d04341c12 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:32.840145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7d378-276c-45e5-9ae1-81408074309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:32.840801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d378-276c-45e5-9ae1-81408074309a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.841089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d378-276c-45e5-9ae1-81408074309a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.841370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d378-276c-45e5-9ae1-81408074309a None admin] PATH_INFO: `/v3/services/network` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.842011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7d378-276c-45e5-9ae1-81408074309a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:32.843450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7d378-276c-45e5-9ae1-81408074309a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:32.852038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 398/815] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:32 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/262558e0503a4f57a619c75acfdae331/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:32.853039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-31d7d378-276c-45e5-9ae1-81408074309a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:32.861616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:32.862725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:42:32.863629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:32.868145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 418/816] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:32 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:32.870171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09166c-8062-4d90-8177-1f43ef029ebc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:32.870640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09166c-8062-4d90-8177-1f43ef029ebc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:32.870930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09166c-8062-4d90-8177-1f43ef029ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:32.879439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:32.894891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8c09166c-8062-4d90-8177-1f43ef029ebc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:32.896731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8c09166c-8062-4d90-8177-1f43ef029ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:32.992113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b24141-ff8b-4e0a-8781-f63210330ba4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.003554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 419/817] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:33.003554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09166c-8062-4d90-8177-1f43ef029ebc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.044094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:33.049376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.050034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.050320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.050603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.059162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.062316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:33.074149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:33.075349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 399/818] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:42:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:33.076475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.128375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09166c-8062-4d90-8177-1f43ef029ebc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.160759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:33.167634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.168570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.169135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.169584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.170577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] RBAC: Adding query filter params (name=network) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:33.171150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.172782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.178516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 420/819] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:33.180025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.212630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95004f-ac27-4109-9216-04c22ebb0fcc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.257481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.258312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.258607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.258883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.259518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.261164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:33.273126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:33.274104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:42:33.275362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:33.279567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 400/820] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 07:42:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:33.280452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.301446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8697e-7075-47f7-860e-1b1da7c7bddb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.341125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a0192-0474-434c-8457-fc866816213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.342190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a0192-0474-434c-8457-fc866816213c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.342776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a0192-0474-434c-8457-fc866816213c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.343248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a0192-0474-434c-8457-fc866816213c None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.344307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a0192-0474-434c-8457-fc866816213c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:33.344816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a0192-0474-434c-8457-fc866816213c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.346747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a0192-0474-434c-8457-fc866816213c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.353082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:33.367248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 421/821] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:42:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:33.368676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a0192-0474-434c-8457-fc866816213c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.403217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e79f3a-8e22-43b2-8b62-edb0c69bc8ab None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.427102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:33.440094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.440486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.440780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.445551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.445551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] RBAC: Adding query filter params (type=network) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:33.445551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.445551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.449248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 401/822] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:33.450235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.477454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a0192-0474-434c-8457-fc866816213c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.514736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.515719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.516179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.516645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.517496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:33.517975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.519801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.526112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 422/823] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 07:42:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:33.527182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.560506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a45f-41cd-49cc-95d3-f502bba9f6ef None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.580890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:33.597433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.597888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.599951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.599951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.640125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a6cfa-49d7-44bc-a78d-ee925b435e87 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.664214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:33.675550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.677082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.684119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.685032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.685490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.686066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.686975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.688673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.773911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 423/824] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:42:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:33.776761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.785987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 402/825] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:42:33.796029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:33.834298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:33.919414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc4d31-3f64-4bac-95de-73280be9d0c4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:33.961296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:33.962560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:33.963072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:33.964118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:33.964851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:33.965603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:33.966582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:33.972549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 424/826] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 07:42:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:33.979511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:34.030491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae1e21-63fa-43b7-940a-49ac510bcf71 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:34.047304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:34.082237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:34.084107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.085124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.085694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.099934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:34.101631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:34.120340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:34.121306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:34.122220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:34.122220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:34.122220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:34.122220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:34.124448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 403/827] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:34.129952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:34.130458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262ee7c-9864-416b-a6fe-287b2af051fa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:34.172857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:34.173226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.173521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.173801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.174524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:34.175950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:34.225177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:34.251811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 425/828] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:42:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:34.260073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072ce7b-e7a5-41cd-a16b-aadeb16e8a9f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:34.309934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:34.311464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.311973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.312428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.313416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:34.313959 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:34.315605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:34.331150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 404/829] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:34.342176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:34.445099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:34.456119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669be36-addb-4569-a1a7-9f5839db8f90 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:34.492135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e1a4f-a09d-4944-95a5-14b52f26f599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:34.492792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e1a4f-a09d-4944-95a5-14b52f26f599 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.493082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e1a4f-a09d-4944-95a5-14b52f26f599 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.493363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e1a4f-a09d-4944-95a5-14b52f26f599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.501161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e1a4f-a09d-4944-95a5-14b52f26f599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:34.502753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e1a4f-a09d-4944-95a5-14b52f26f599 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f30e1a4f-a09d-4944-95a5-14b52f26f599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:34.516150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:34.517107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:34.518099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 426/830] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:34.524936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:34.633456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d9718-93aa-4e48-bd0d-1577f10c25b9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:34.661659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:34.671239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:34.672181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.672650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.673105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.674036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:34.674530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:34.676202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:34.681058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6807a-90b5-47c1-90dd-cf26265833f6 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.681356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6807a-90b5-47c1-90dd-cf26265833f6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.681642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6807a-90b5-47c1-90dd-cf26265833f6 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.683365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 427/831] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:34.688692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bc8dd-3c08-4ff3-8e8e-819166325816 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.689361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bc8dd-3c08-4ff3-8e8e-819166325816 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.689361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bc8dd-3c08-4ff3-8e8e-819166325816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.693040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 405/832] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:34.700246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:34.710763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0a6bc8dd-3c08-4ff3-8e8e-819166325816 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:34.712202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0a6bc8dd-3c08-4ff3-8e8e-819166325816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:34.803581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 428/833] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:34.819862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bc8dd-3c08-4ff3-8e8e-819166325816 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:34.833748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc79083-78ae-4a3b-92dd-752d99b33c57 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:34.878637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:34.879718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.880108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.880492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:34.881643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=659b367f1ce643a68d55ff1c00d8b7b8) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:34.882061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:34.884481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:34.892317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 406/834] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:34 2022] GET /identity/v3/role_assignments?user.id=659b367f1ce643a68d55ff1c00d8b7b8&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:34.931049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:34.953350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bc8dd-3c08-4ff3-8e8e-819166325816 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:34.992999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f18b3e-9676-4025-b868-6f71ec40b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:34.993666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18b3e-9676-4025-b868-6f71ec40b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:34.994008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18b3e-9676-4025-b868-6f71ec40b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:34.994292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18b3e-9676-4025-b868-6f71ec40b5b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.007124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f18b3e-9676-4025-b868-6f71ec40b5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f5f18b3e-9676-4025-b868-6f71ec40b5b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:35.012533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:35.013557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:35.014439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 429/835] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:42:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:35.021786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.038355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721fa6-b0ac-4df1-bd40-3ed4dfeb4308 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.038901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721fa6-b0ac-4df1-bd40-3ed4dfeb4308 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.039336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721fa6-b0ac-4df1-bd40-3ed4dfeb4308 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.041140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 430/836] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:35.047874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3d332-0b07-4d5c-8784-2ff4a813eba9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.048581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3d332-0b07-4d5c-8784-2ff4a813eba9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.049030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3d332-0b07-4d5c-8784-2ff4a813eba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.074979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0be3d332-0b07-4d5c-8784-2ff4a813eba9 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:35.076908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0be3d332-0b07-4d5c-8784-2ff4a813eba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:35.091666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:35.148196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d70828-3e6d-42c3-8d54-c14b7403d806 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.164600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 431/837] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:35.188167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3d332-0b07-4d5c-8784-2ff4a813eba9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.189525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.190637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.190921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.193579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.193579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:35.195917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.197585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:35.208459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 407/838] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:35.215647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.275566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:35.317306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3d332-0b07-4d5c-8784-2ff4a813eba9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.348706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321007a-5d21-40a3-8e4a-34f4d9f19260 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.356636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.357507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.357966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.358401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.365645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:35.371616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:35.377549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:35.378528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:42:35.379392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:35.387517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 432/839] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:35.388982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.392294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.393049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.393409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.394499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.406220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.407857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:35.424056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:35.425200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:35.426317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:35.426317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:35.426317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:35.426317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:35.432466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 408/840] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:35.436635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.519045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f345-3eb6-41a4-aaaf-d039bab7b3f2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.556874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:35.565131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.566281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.566570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.573161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.573161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:35.573161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.573161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:35.573683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d67cb-4422-4f9a-92e7-3862c15e491c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.586703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 433/841] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:35.590706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.617115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.617864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.618397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.619110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:35.619506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.621180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:35.638919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 409/842] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:35.639990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.670492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:35.703737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb902-8d08-446f-8945-f57adef321a0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.741616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.742720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.743282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.743789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.749778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a6f9b-82f8-457e-9fc9-b4539649f770 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.763169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.764888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:35.781900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:35.783060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:35.784199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:35.784199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:35.784199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:35.784199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:35.789962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 434/843] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:35.792238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.795802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.796758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.797221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.797675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.805648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.807970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:35.820831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:35.821786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:35.822803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:35.830072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 410/844] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:35.831193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.844786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:35.906425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031f35-8d0a-42cf-87e3-773fabe32a8a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.927902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:35.944266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.944968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.947549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.947549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.947834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6320c-e47c-44a2-8469-42a17b6809d3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:35.948003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:35.948003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.948279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:35.966743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 435/845] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:35.968052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:35.986940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:35.987615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:35.987886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:35.988164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:35.988957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:35.989270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:35.990702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.013854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 411/846] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:36.014781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.115404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a977049-c18a-4123-b925-7293d95263b8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.119184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.123039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f32de-8db4-4ff9-ac7a-a92b22224f9d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.163545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.164213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.164506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.164788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.169856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e002ff-b4c2-4d26-bb83-e5cd8bed647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.170676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e002ff-b4c2-4d26-bb83-e5cd8bed647c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.171946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e002ff-b4c2-4d26-bb83-e5cd8bed647c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.171946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e002ff-b4c2-4d26-bb83-e5cd8bed647c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.173464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.175172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.189636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:36.191064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:36.192041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:42:36.192994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:36.195610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 436/847] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:42:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:36.199761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.262893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e002ff-b4c2-4d26-bb83-e5cd8bed647c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.264883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e002ff-b4c2-4d26-bb83-e5cd8bed647c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.324820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c96b7-3745-42be-b9ec-0c32a2662640 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.367676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2495e4-68a6-4799-95dd-d2facdd976ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.368307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2495e4-68a6-4799-95dd-d2facdd976ab None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.368595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2495e4-68a6-4799-95dd-d2facdd976ab None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.368876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2495e4-68a6-4799-95dd-d2facdd976ab None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.370225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2495e4-68a6-4799-95dd-d2facdd976ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:36.370592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2495e4-68a6-4799-95dd-d2facdd976ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.372012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2495e4-68a6-4799-95dd-d2facdd976ab None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.374814 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 412/848] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:42:36.376646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b531ab-b756-4442-bd12-9715b9379395 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.377153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b531ab-b756-4442-bd12-9715b9379395 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.377636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b531ab-b756-4442-bd12-9715b9379395 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.379316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 413/849] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:36.380597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b531ab-b756-4442-bd12-9715b9379395 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.397008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 437/850] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:36.397008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a7154-ca1f-4685-b9b5-cd2132de4632 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.397008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a7154-ca1f-4685-b9b5-cd2132de4632 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.397008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a7154-ca1f-4685-b9b5-cd2132de4632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.412675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c30a7154-ca1f-4685-b9b5-cd2132de4632 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:36.413873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c30a7154-ca1f-4685-b9b5-cd2132de4632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:36.431283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.436964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.484968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 438/851] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:36.485739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a7154-ca1f-4685-b9b5-cd2132de4632 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.497142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b531ab-b756-4442-bd12-9715b9379395 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.534778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.535435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.535768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.536048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.548645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.550102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:36.566535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:36.567461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:36.568533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:36.568533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:36.568533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:36.568533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:36.572282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 414/852] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:36.573088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.605469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a7154-ca1f-4685-b9b5-cd2132de4632 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.644231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.645161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.645614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.646148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.668987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.674522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.685493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1188a68-f55b-49bf-a5e9-043ea5e02f66 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.715267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.716048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.716333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.716655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.717353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.718720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.719759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.721225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:36.728552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:36.729571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:36.730418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:36.733087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 415/853] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:36.734053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.833715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 439/854] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:42:36.834583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.857992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13eb01-0c52-4135-90ca-6ceb31a20ed1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.887262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.895197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.902661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.902661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.902661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.902661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:36.902661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.902661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.915834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 416/855] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:36.915834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:36.937410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:36.941024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712f68-b437-4cec-b8ba-45986a7374c8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:36.979720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:36.980367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:36.980609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:36.980975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:36.981562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:36.981866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:36.983230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:36.993058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 440/856] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:36.993058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.030700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41029-388b-499e-9291-f28c369b7419 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.068916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.069562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.069847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.070123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.080269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.081722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.094840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112b84b-5990-4ee4-806a-d5b9341fbaa5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:37.096661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:37.097763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:37.103486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:37.103486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:37.103486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:37.103486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:37.103486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 417/857] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:37.103486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.111716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:37.129621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b437-9dfd-4015-b601-d33b89cf3d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.130260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b437-9dfd-4015-b601-d33b89cf3d40 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.130540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b437-9dfd-4015-b601-d33b89cf3d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.130807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b437-9dfd-4015-b601-d33b89cf3d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.138232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b437-9dfd-4015-b601-d33b89cf3d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.139727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b437-9dfd-4015-b601-d33b89cf3d40 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-9af6b437-9dfd-4015-b601-d33b89cf3d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:37.151013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:37.152119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 441/858] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:37.153452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6a00d-e114-4e25-a06f-ceb697906f83 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.154015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6a00d-e114-4e25-a06f-ceb697906f83 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.154015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6a00d-e114-4e25-a06f-ceb697906f83 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.155316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 442/859] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:37.159853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6a00d-e114-4e25-a06f-ceb697906f83 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.174436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:37.204683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26302fd8-2e20-4b07-9a10-e80f918255e8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.239000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.239653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.239938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.240224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.240905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:37.241215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.242621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.247732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 418/860] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:37.248405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.276735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6a00d-e114-4e25-a06f-ceb697906f83 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.319114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad682ffd-e2bf-4f39-bdc6-ca8d876abc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.319696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad682ffd-e2bf-4f39-bdc6-ca8d876abc15 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.319988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad682ffd-e2bf-4f39-bdc6-ca8d876abc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.320269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad682ffd-e2bf-4f39-bdc6-ca8d876abc15 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.321100 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad682ffd-e2bf-4f39-bdc6-ca8d876abc15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:37.321415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad682ffd-e2bf-4f39-bdc6-ca8d876abc15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.322972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad682ffd-e2bf-4f39-bdc6-ca8d876abc15 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.341328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 443/861] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:37.342801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44df33-c618-4031-99df-3cca62c97ab7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.343101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44df33-c618-4031-99df-3cca62c97ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.343505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44df33-c618-4031-99df-3cca62c97ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.364782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612b0b8-5279-44f6-9d5c-a9837edb7591 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.365929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ee44df33-c618-4031-99df-3cca62c97ab7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:37.367179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ee44df33-c618-4031-99df-3cca62c97ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:37.400043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:37.405965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.407951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.407951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.409720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.409720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:37.409720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.411088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.414169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:37.426811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 419/862] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:37.427701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.448400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 444/863] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:37.449641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44df33-c618-4031-99df-3cca62c97ab7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.537298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139781f5-83a1-44db-a15f-b3e159a41ded None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.565479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44df33-c618-4031-99df-3cca62c97ab7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.573298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.573913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.574196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.574469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.581933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.583640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:37.598873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:37.599830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:37.600767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 420/864] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:37.601894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.605047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.605916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.606203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.606476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.607274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=3001dbc46cc441cfb61b1d63a4461314) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:37.607611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.609051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.615259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 445/865] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/role_assignments?user.id=3001dbc46cc441cfb61b1d63a4461314&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:37.623748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.662364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:37.675651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:37.768591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89de0b-9a9c-4ea4-bc57-40254e086fb9 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.796830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a30772-f969-4e61-bb36-476488b44c95 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:37.819648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.820926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.821422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.821942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.823807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:37.824862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.826796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.840228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:37.840880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:37.841172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:37.841456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:37.842060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:37.843432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:37.844528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 446/866] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:37.850639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:37.855441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:37.856417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:37.857280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:37.858589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 421/867] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:37.870616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:38.001746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:38.012674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d6cf8-0b04-417b-8927-4afbba5c1c67 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:38.052939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e1220-5566-4574-b9d0-107ecec663b6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:38.077019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:38.084278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7395fe9-9501-40a8-b834-5e7c5f9cc208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:38.084905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395fe9-9501-40a8-b834-5e7c5f9cc208 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:38.085178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395fe9-9501-40a8-b834-5e7c5f9cc208 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:38.085453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395fe9-9501-40a8-b834-5e7c5f9cc208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:38.086259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395fe9-9501-40a8-b834-5e7c5f9cc208 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=262558e0503a4f57a619c75acfdae331) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:38.086572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395fe9-9501-40a8-b834-5e7c5f9cc208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:38.089050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395fe9-9501-40a8-b834-5e7c5f9cc208 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:38.094229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:38.095079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:38.095522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:38.099083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:38.099083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:38.099083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:38.099083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:38.104007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 422/868] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:38.109179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 447/869] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:37 2022] GET /identity/v3/role_assignments?user.id=262558e0503a4f57a619c75acfdae331&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:38.110643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:38.273805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a48c4-dad8-4c0f-96ee-8f8b2dbe4e51 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:38.329941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507af008-f7c5-4b04-9344-be0a7c49f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:38.330496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507af008-f7c5-4b04-9344-be0a7c49f548 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:38.330779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507af008-f7c5-4b04-9344-be0a7c49f548 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:38.331054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507af008-f7c5-4b04-9344-be0a7c49f548 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:38.331770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507af008-f7c5-4b04-9344-be0a7c49f548 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:38.332079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507af008-f7c5-4b04-9344-be0a7c49f548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:38.333463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507af008-f7c5-4b04-9344-be0a7c49f548 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:38.340367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 423/870] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:39.346268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f3684-ce8c-4107-bf75-434f7a1a3a13 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:39.347289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f3684-ce8c-4107-bf75-434f7a1a3a13 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:39.347803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f3684-ce8c-4107-bf75-434f7a1a3a13 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:39.349612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 448/871] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:39.356023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256ce6e-4ea8-4e35-b759-66272fa62ca9 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:39.356923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256ce6e-4ea8-4e35-b759-66272fa62ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:39.357404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256ce6e-4ea8-4e35-b759-66272fa62ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:39.380120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7256ce6e-4ea8-4e35-b759-66272fa62ca9 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:39.381677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7256ce6e-4ea8-4e35-b759-66272fa62ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:39.432757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:39.460201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 424/872] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:39.484505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f3684-ce8c-4107-bf75-434f7a1a3a13 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:39.604741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f3684-ce8c-4107-bf75-434f7a1a3a13 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:39.641612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17d2f1-b4ef-4173-bcb3-feacfe757c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:39.642599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17d2f1-b4ef-4173-bcb3-feacfe757c2b None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:39.643085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17d2f1-b4ef-4173-bcb3-feacfe757c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:39.643586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17d2f1-b4ef-4173-bcb3-feacfe757c2b None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:39.644479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17d2f1-b4ef-4173-bcb3-feacfe757c2b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:39.646059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17d2f1-b4ef-4173-bcb3-feacfe757c2b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:39.706999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 449/873] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:39 2022] POST /identity/v3/services => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:39.904082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2879db-1792-46a1-8e68-58ce197e3a8b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:39.904433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2879db-1792-46a1-8e68-58ce197e3a8b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:39.904624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2879db-1792-46a1-8e68-58ce197e3a8b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:39.906299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 425/874] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:39.911632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d818e-635f-4fde-9d3b-eac0a4ee26a5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:39.911927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d818e-635f-4fde-9d3b-eac0a4ee26a5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:39.912207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d818e-635f-4fde-9d3b-eac0a4ee26a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:39.931974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e84d818e-635f-4fde-9d3b-eac0a4ee26a5 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:39.933653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e84d818e-635f-4fde-9d3b-eac0a4ee26a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:39.944523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:40.019645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 450/875] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:40.036519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2879db-1792-46a1-8e68-58ce197e3a8b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.144106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2879db-1792-46a1-8e68-58ce197e3a8b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:40.180684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:40.181261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.181655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.182017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.193073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:40.198687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:40.199693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:40.200564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 426/876] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:40.206121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d818e-635f-4fde-9d3b-eac0a4ee26a5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.314875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d818e-635f-4fde-9d3b-eac0a4ee26a5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:40.351680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70a1be-a055-44cd-8f9f-5c60ba53c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:40.353142 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a1be-a055-44cd-8f9f-5c60ba53c842 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.354172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a1be-a055-44cd-8f9f-5c60ba53c842 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.354654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a1be-a055-44cd-8f9f-5c60ba53c842 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.355599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70a1be-a055-44cd-8f9f-5c60ba53c842 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:40.356113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70a1be-a055-44cd-8f9f-5c60ba53c842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:40.357713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70a1be-a055-44cd-8f9f-5c60ba53c842 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:40.368265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 451/877] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:40.376882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.390595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4620c2-60b4-4800-a947-5862ef793f6a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.390899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4620c2-60b4-4800-a947-5862ef793f6a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.391189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4620c2-60b4-4800-a947-5862ef793f6a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.393276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 452/878] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:40.398342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827fe5-5f34-467d-88cc-210c3c2511d2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.398633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827fe5-5f34-467d-88cc-210c3c2511d2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.398981 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827fe5-5f34-467d-88cc-210c3c2511d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.405928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:40.422534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0e827fe5-5f34-467d-88cc-210c3c2511d2 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:40.423959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0e827fe5-5f34-467d-88cc-210c3c2511d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:40.425347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:40.492133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acc5fc-6e4d-4279-a01f-4854c04bf9b7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:40.504458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 453/879] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:40.505791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0a8f6-ef28-4e97-ba6a-21dfb04e13ff None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.506078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0a8f6-ef28-4e97-ba6a-21dfb04e13ff None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.506347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0a8f6-ef28-4e97-ba6a-21dfb04e13ff None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.511636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 454/880] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:40.512787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782783e-ab8f-4671-a8b5-8d473cb0e5c3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.513069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782783e-ab8f-4671-a8b5-8d473cb0e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.513344 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782783e-ab8f-4671-a8b5-8d473cb0e5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.534616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:40.534953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8782783e-ab8f-4671-a8b5-8d473cb0e5c3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:40.535266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.535599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.535860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.536069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8782783e-ab8f-4671-a8b5-8d473cb0e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:40.547416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:40.548909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:40.564340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:40.565285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:40.566196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:40.566196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:40.566196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:40.566196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:40.566196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 427/881] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:40.566475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.614942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 455/882] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:40.616472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782783e-ab8f-4671-a8b5-8d473cb0e5c3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.665627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:40.677675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476d112-1767-4019-8516-5a69bdf3c575 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:40.687614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:40.714063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:40.714702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.714982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.715256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.715905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:40.717264 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:40.726236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:40.727231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:40.734206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:40.734989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782783e-ab8f-4671-a8b5-8d473cb0e5c3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:40.735065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 428/883] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:40.735065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.771520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:40.772171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.772454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.773403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.773403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:40.773700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:40.775073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:40.791696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 456/884] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:40.791696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.846564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043921c2-f7d8-4c34-9a18-8762ef250ec7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:40.883117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:40.883762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.884046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.884315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.885649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:40.887070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:40.897990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160f2d1-5e2b-4f1a-9040-9234d68c6e16 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:40.923912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:40.935604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:40.936240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:40.936531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:40.936811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:40.937522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:40.937832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:40.939210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:40.943798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 457/885] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:40.944573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:40.973922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 429/886] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:40 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:40.974899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:41.016248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:41.078788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64bd39-146c-477b-ad66-7e37066508ee None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:41.130416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:41.131210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:41.131501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:41.132694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:41.133290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4c95f-dad2-45bb-8c14-92fd5de62c8e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:41.141605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:41.143030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:41.157532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:41.158504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:41.159451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 458/887] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:41.164604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:41.172766 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:41.173400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:41.173701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:41.173961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:41.174647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:41.175061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:41.176469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:41.182243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 430/888] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:41.225549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:41.294049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bea137-dbb5-4759-b2e1-b034c8e470c6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:41.334565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:41.335271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:41.335688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:41.335979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:41.337059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:41.337373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:41.339401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:41.359865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 459/889] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 07:42:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:41.365583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:41.454418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:41.477864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48039f0-2b5b-4d89-899a-0ea56b064393 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:41.518473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b92abc-f290-44db-b758-136fafe27c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:41.519103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b92abc-f290-44db-b758-136fafe27c83 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:41.519389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b92abc-f290-44db-b758-136fafe27c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:41.519693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b92abc-f290-44db-b758-136fafe27c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:41.593085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b92abc-f290-44db-b758-136fafe27c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:41.594467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b92abc-f290-44db-b758-136fafe27c83 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:41.678425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:41.702409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 431/890] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:41 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:42:41.708141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:41.811583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:41.816080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4fe38-4d89-41f8-91fc-2b062983b825 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:41.852668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:41.853276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:41.853731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:41.854234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:41.857070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510133-32ee-438e-a738-4b1301261e3a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:41.857360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510133-32ee-438e-a738-4b1301261e3a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:41.857636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15510133-32ee-438e-a738-4b1301261e3a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:41.859209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 432/891] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:41.864803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b054c-d944-4bd8-aa10-ece159c45667 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:41.865201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b054c-d944-4bd8-aa10-ece159c45667 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:41.865683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b054c-d944-4bd8-aa10-ece159c45667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:41.870305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:41.871923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:41.886115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-db6b054c-d944-4bd8-aa10-ece159c45667 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:41.887449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-db6b054c-d944-4bd8-aa10-ece159c45667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:41.891238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:41.892321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:41.893429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:41.893429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:41.893429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:41.893429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:41.893429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 460/892] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:41.899630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:41.969596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 433/893] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:41.991783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b054c-d944-4bd8-aa10-ece159c45667 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.039083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563141c-6405-4cea-98ef-bd96a59238ed None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.085828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.086703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.086999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.087279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.088586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:42.088899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.090914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.107882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 461/894] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:42.113966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.125616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b054c-d944-4bd8-aa10-ece159c45667 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.133287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:42.136470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:42.164213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.165151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.165659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.166963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.168024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.169762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:42.180344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:42.188274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:42.189323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 434/895] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:42.190811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.248327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00bd98-623b-4074-abe3-5a2f1de345b7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.285993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.287001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.287471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.287970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.295343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.297079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:42.309359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:42.310378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:42.311291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:42.318275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 462/896] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:42.319779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.328194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e503796-4af7-4f7e-9208-70c1be65eda0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.377654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.378313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.378601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.378878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.379582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:42.379898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.381800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.391609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 435/897] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:42.394723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.407412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:42.438551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b548e-d589-4aab-8fd1-2d086a06733a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.444144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:42.479087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.479857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.480262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.480641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.481704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:42.482262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.483826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.503332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 463/898] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:42.505088 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.529490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f83368-7c3a-4857-bbfb-b49cef4989fc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.569149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.570003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.570386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.570761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.571555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:42.572519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.574145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.584071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 436/899] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:42.585183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.635408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b62542-493e-4d73-aebe-16fcf427b874 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.668585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:42.674548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.684658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.684658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.684658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.684658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] RBAC: Adding query filter params (role.id=1d2b3f7e9adb449db4663a3dba84a1a4, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=3001dbc46cc441cfb61b1d63a4461314) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:42.684658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.684658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.691881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 464/900] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:42 2022] GET /identity/v3/role_assignments?user.id=3001dbc46cc441cfb61b1d63a4461314&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=1d2b3f7e9adb449db4663a3dba84a1a4 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:42.720462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:42.724383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa7f17-0409-49fc-af20-336f373f594b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.765617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13df4f2-b4fb-46f0-95d2-44eac0d8389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.766635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df4f2-b4fb-46f0-95d2-44eac0d8389a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.767126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df4f2-b4fb-46f0-95d2-44eac0d8389a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.767643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df4f2-b4fb-46f0-95d2-44eac0d8389a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.768682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13df4f2-b4fb-46f0-95d2-44eac0d8389a None admin] RBAC: Adding query filter params (interface=public, service_id=bcba4a834bda48a4bd30cce16eb839eb, region_id=RegionOne) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:42.769231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13df4f2-b4fb-46f0-95d2-44eac0d8389a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.770995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13df4f2-b4fb-46f0-95d2-44eac0d8389a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.780662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 437/901] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:42:42 2022] GET /identity/v3/endpoints?service_id=bcba4a834bda48a4bd30cce16eb839eb&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:42.786293 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:42.899019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468ee56-9078-4933-a245-26995266f2f4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:42.935518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dc8cd-3e5c-48ef-abfa-04d25cc69b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:42.936176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dc8cd-3e5c-48ef-abfa-04d25cc69b05 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.936467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dc8cd-3e5c-48ef-abfa-04d25cc69b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.936750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dc8cd-3e5c-48ef-abfa-04d25cc69b05 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.937413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dc8cd-3e5c-48ef-abfa-04d25cc69b05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:42.938801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dc8cd-3e5c-48ef-abfa-04d25cc69b05 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:42.945070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 465/902] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:42:42 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:42:43.104110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c340f53-a4c2-4933-bce8-223f15ba5902 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.104493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c340f53-a4c2-4933-bce8-223f15ba5902 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.104672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c340f53-a4c2-4933-bce8-223f15ba5902 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.106293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 438/903] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:43.111893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29890011-2917-4982-b5c4-31f11c54354a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.112190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29890011-2917-4982-b5c4-31f11c54354a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.112474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29890011-2917-4982-b5c4-31f11c54354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.132056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-29890011-2917-4982-b5c4-31f11c54354a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:43.133274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-29890011-2917-4982-b5c4-31f11c54354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:43.193483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:43.207566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 466/904] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:43.230650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c340f53-a4c2-4933-bce8-223f15ba5902 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:43.290460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365b985-8d0b-4de7-ae42-d66494966268 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.291294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365b985-8d0b-4de7-ae42-d66494966268 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.291574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365b985-8d0b-4de7-ae42-d66494966268 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.293403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 467/905] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:43.298521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39e35b-84eb-422a-9e15-9d3d0f37bff7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.298809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39e35b-84eb-422a-9e15-9d3d0f37bff7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.299090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39e35b-84eb-422a-9e15-9d3d0f37bff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.320461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ee39e35b-84eb-422a-9e15-9d3d0f37bff7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:43.322751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ee39e35b-84eb-422a-9e15-9d3d0f37bff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:43.342440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c340f53-a4c2-4933-bce8-223f15ba5902 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:43.385991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:43.386695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.386988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.387267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.388456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:43.390171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:43.403282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:43.404289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:43.405143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:43.405956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 468/906] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:43.407689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 439/907] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 07:42:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:43.411928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39e35b-84eb-422a-9e15-9d3d0f37bff7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:43.431715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:43.475420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:43.539129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39e35b-84eb-422a-9e15-9d3d0f37bff7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:43.559724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86830c1f-1b3a-4955-a2eb-3315eb124e39 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:43.566741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:43.586061 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:43.587388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.588218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.588680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.590115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:43.590681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:43.593526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:43.597748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:43.598501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.598889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.599267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.600108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:43.600266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 469/908] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:43.601692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:43.605453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:43.676678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 440/909] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:43 2022] POST /identity/v3/services => generated 241 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:42:43.720559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ba488-71b8-4300-bdf7-ba707293e9c6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:43.781396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:43.782037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.782319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.782592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.783283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:43.783622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:43.785029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:43.791593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 470/910] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:43.797686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:43.896538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:43.909913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e8e62-7099-48fa-aff4-926d61f3feb0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:43.946145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb018d7-f9be-4542-b006-16ec99799fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:43.946771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb018d7-f9be-4542-b006-16ec99799fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.947591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb018d7-f9be-4542-b006-16ec99799fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.947922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb018d7-f9be-4542-b006-16ec99799fa5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.948673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb018d7-f9be-4542-b006-16ec99799fa5 None admin] RBAC: Adding query filter params (interface=public, service_id=42b38cf1ac61448198d830eedbd4d830, region_id=RegionOne) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:43.948983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb018d7-f9be-4542-b006-16ec99799fa5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:43.950347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb018d7-f9be-4542-b006-16ec99799fa5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:43.956049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 441/911] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:42:43 2022] GET /identity/v3/endpoints?service_id=42b38cf1ac61448198d830eedbd4d830&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:43.961813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:44.011493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:44.067984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550d8c-2c5a-4254-8c4d-43ab3430df1b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:44.104893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9bccf-90f5-45af-8baa-239ecd015a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:44.105698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bccf-90f5-45af-8baa-239ecd015a59 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:44.106003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bccf-90f5-45af-8baa-239ecd015a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:44.106250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bccf-90f5-45af-8baa-239ecd015a59 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:44.106901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9bccf-90f5-45af-8baa-239ecd015a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:44.108305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9bccf-90f5-45af-8baa-239ecd015a59 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:44.114638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 471/912] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:42:43 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:42:44.800418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30a6d-0251-4a0f-971c-56f2b390733a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:44.800967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30a6d-0251-4a0f-971c-56f2b390733a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:44.800967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30a6d-0251-4a0f-971c-56f2b390733a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:44.802585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 442/913] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:44.808536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22076a7-a1ea-4705-bbeb-c9df3a8aa077 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:44.809084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22076a7-a1ea-4705-bbeb-c9df3a8aa077 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:44.809084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22076a7-a1ea-4705-bbeb-c9df3a8aa077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:44.827515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c22076a7-a1ea-4705-bbeb-c9df3a8aa077 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:44.828742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c22076a7-a1ea-4705-bbeb-c9df3a8aa077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:44.905538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 472/914] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:44.927273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30a6d-0251-4a0f-971c-56f2b390733a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.033885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30a6d-0251-4a0f-971c-56f2b390733a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.070056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.070279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39fa3b-c6b2-4901-8e81-37bc694112ce None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.070570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39fa3b-c6b2-4901-8e81-37bc694112ce None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.070850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39fa3b-c6b2-4901-8e81-37bc694112ce None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.071606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.072641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.073113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.076691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 473/915] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:45.078085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9040c-a0ce-41fd-90e9-0de38d8455d4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.078375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9040c-a0ce-41fd-90e9-0de38d8455d4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.078650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9040c-a0ce-41fd-90e9-0de38d8455d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.085061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:45.090645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:45.091652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:42:45.092535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:45.095149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 443/916] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 07:42:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:45.103564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7ef9040c-a0ce-41fd-90e9-0de38d8455d4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:45.103564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7ef9040c-a0ce-41fd-90e9-0de38d8455d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:45.103775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.116504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:45.136163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:45.191638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 474/917] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:45.211819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9040c-a0ce-41fd-90e9-0de38d8455d4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.223642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b8517-2d9e-4673-a017-62d59d81ea00 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.267518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.268349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.268828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.269121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.269845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:45.270202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.271751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.282197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 444/918] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:45.288899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.332515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9040c-a0ce-41fd-90e9-0de38d8455d4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.372039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.372798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.373144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.373562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.374194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.376101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.386539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:45.387131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:45.388252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:45.389304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 475/919] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:45.394800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.424532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af6db0-1b1b-4dbd-b894-1b5bf87725a9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.432206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:45.463042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.464169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.464718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.465643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.478616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.480606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:45.498126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:45.499070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:45.500024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:45.500024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:45.500024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:45.500024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:45.502359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 445/920] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:45.507986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.514926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14edafa-7f5c-4ab3-8816-a96be3eb3e3e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.551481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.552070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.552389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.552967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.553435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:45.553749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.555150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.563564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 476/921] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:45.568362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.627152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87df20-ce0e-41e1-ad7f-5b09d45070a5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.656883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:45.668183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.669900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.669900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.669900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.670533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:45.670940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.672441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.692248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 446/922] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:45.693649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.700090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:45.707106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a72a7-423b-4013-b0e2-434067076dfc None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.747095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.747973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.748631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.748631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.749604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:45.750025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.751570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.761957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 477/923] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:45.763923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.822519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e35c8-75b4-45f0-8ec9-e55772560af0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.859024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec711f-f525-4156-960a-f3be532b9e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.859708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec711f-f525-4156-960a-f3be532b9e82 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.860092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec711f-f525-4156-960a-f3be532b9e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.860268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec711f-f525-4156-960a-f3be532b9e82 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.867830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec711f-f525-4156-960a-f3be532b9e82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.869606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec711f-f525-4156-960a-f3be532b9e82 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.880287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bbc85-e065-4bbb-9008-ebffd67a58e5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2dec711f-f525-4156-960a-f3be532b9e82 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:45.882228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:45.883165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:45.884208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 447/924] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:45.886116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72594095-35d7-4535-8f52-fc7249d80ee6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.886116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72594095-35d7-4535-8f52-fc7249d80ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.886116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72594095-35d7-4535-8f52-fc7249d80ee6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.886930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 448/925] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:45.891421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72594095-35d7-4535-8f52-fc7249d80ee6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:45.921020 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22604e-0b7a-4667-878c-b1be900efc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:45.921700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22604e-0b7a-4667-878c-b1be900efc07 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:45.921993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22604e-0b7a-4667-878c-b1be900efc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:45.922281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22604e-0b7a-4667-878c-b1be900efc07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:45.922992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22604e-0b7a-4667-878c-b1be900efc07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:45.924408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22604e-0b7a-4667-878c-b1be900efc07 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:45.943812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:45.999016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72594095-35d7-4535-8f52-fc7249d80ee6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.004038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 478/926] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:42:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:42:46.013705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8594f4c-bf10-4fc4-b1a8-8ff78a3171bc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.014086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8594f4c-bf10-4fc4-b1a8-8ff78a3171bc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.014370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8594f4c-bf10-4fc4-b1a8-8ff78a3171bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.046123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.047022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.047553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.047983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.049196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:46.049666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.051362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.065284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d8594f4c-bf10-4fc4-b1a8-8ff78a3171bc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:46.066507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d8594f4c-bf10-4fc4-b1a8-8ff78a3171bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:46.078614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 449/927] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 07:42:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:46.079863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.086608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:46.171948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 479/928] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:46.189156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d859-b03f-4cbb-a287-9887916c0bfc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.189585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d859-b03f-4cbb-a287-9887916c0bfc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.189936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d859-b03f-4cbb-a287-9887916c0bfc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.191307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 480/929] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:46.196641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e96784-3a9c-4772-bc7b-471748a7eebf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.196945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e96784-3a9c-4772-bc7b-471748a7eebf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.197222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e96784-3a9c-4772-bc7b-471748a7eebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.206840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:46.210497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db61b2d-e4c6-4a26-9c01-dc410a4f15e3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.216584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-98e96784-3a9c-4772-bc7b-471748a7eebf None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:46.217800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-98e96784-3a9c-4772-bc7b-471748a7eebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:46.250960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.251663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.252143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.252253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.265819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.267277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:46.284088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:46.285028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:46.285947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:46.285947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:46.285947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:46.285947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.285947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 450/930] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:46.286719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.303572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 481/931] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:46.303572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e96784-3a9c-4772-bc7b-471748a7eebf None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.395657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:46.414083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad9a02-18a9-4a20-9652-d514e611877d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.420662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e96784-3a9c-4772-bc7b-471748a7eebf None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.450953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.451636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.451934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.452212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.452845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.454220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.458707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.459484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.459933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.460349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.461437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:46.461885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.463592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:46.463729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:46.464857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.465724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 451/932] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:46.466534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.489714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 482/933] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:46.489904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:46.496732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.586051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73443a-0ca0-46c7-b0a3-1a60c47964a6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.606807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10552baf-264e-49d7-b7f9-caa392804041 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.624157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.624800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.625082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.625356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.625979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.627461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:46.637488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:46.638641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.639876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 452/934] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:46.640775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.646818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.647391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.647718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.648111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.648803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:46.649113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.651029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.652959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:46.661279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 483/935] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:46.661279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.737183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:46.760269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f6892-d939-40de-b272-2a68f6c5c73f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.777564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4003580-359d-4610-a350-7289c9e55d87 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.798006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.798658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.798934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.799205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.806267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.807775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.813906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.814535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.814823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.815102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.815822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:46.816133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.817505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:46.819229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:46.820306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:46.821519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:46.826010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 453/936] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:46.826472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 484/937] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:46.826841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.827697 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.901818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:46.939802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802b5dc-4e92-400a-b39a-132598dd66a2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.942202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674385f1-30bd-4d3b-b036-cc01c14165d7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:46.975673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.976293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.976578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.976850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.977532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:46.977654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:46.978373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.978805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:46.979093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:46.979376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:46.979715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.980160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:46.980474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:46.981904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:46.985003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 454/938] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:46.985972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:46.994758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:47.001335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 485/939] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:47.001335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:47.098152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18aacb-3b17-4ad6-b9e8-482e99d13116 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:47.111815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ca6b-51a7-4772-8733-99b751af1cd4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:47.134609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:47.135240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:47.135523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:47.135882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:47.136347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:47.136567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:47.136875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:47.138258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:47.148248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:47.148404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 455/940] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:47.148769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:47.149063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:47.149228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:47.149380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:47.150643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] RBAC: Adding query filter params (interface=public, service_id=71389e780622464bbe8c53b2b9dd3195, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:47.151043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:47.152534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:47.162267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 486/941] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:42:46 2022] GET /identity/v3/endpoints?service_id=71389e780622464bbe8c53b2b9dd3195&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:47.162267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:47.227025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:47.262246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506fa8e-268c-486d-bac5-eae831f4f277 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:47.268452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7c8a6-7aa1-4976-b774-0ef56b8332f2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:47.299996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:47.300725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:47.300960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:47.301238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/3001dbc46cc441cfb61b1d63a4461314/roles/1d2b3f7e9adb449db4663a3dba84a1a4` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:47.303654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=3001dbc46cc441cfb61b1d63a4461314, role_id=1d2b3f7e9adb449db4663a3dba84a1a4)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:47.305100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:47.306553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef2c6f-59bb-4fb4-8181-01684d03c1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:47.307177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef2c6f-59bb-4fb4-8181-01684d03c1e2 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:47.307514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef2c6f-59bb-4fb4-8181-01684d03c1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:47.307826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef2c6f-59bb-4fb4-8181-01684d03c1e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:47.308543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef2c6f-59bb-4fb4-8181-01684d03c1e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:47.310494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef2c6f-59bb-4fb4-8181-01684d03c1e2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:47.421509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 456/942] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:42:47 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/3001dbc46cc441cfb61b1d63a4461314/roles/1d2b3f7e9adb449db4663a3dba84a1a4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:47.422264 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:47.450369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 487/943] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:42:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:47.576609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d64d-75ef-42d8-aff1-18654ae62e5f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:47.613370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:47.628684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fc994-27ff-494c-8050-efa3938f670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:47.629318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc994-27ff-494c-8050-efa3938f670c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:47.629604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc994-27ff-494c-8050-efa3938f670c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:47.629948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc994-27ff-494c-8050-efa3938f670c None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:47.630539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fc994-27ff-494c-8050-efa3938f670c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:47.631999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fc994-27ff-494c-8050-efa3938f670c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:47.639561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 457/944] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:42:47 2022] GET /identity/v3/services => generated 1497 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:42:48.116140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b02bc-9165-4dae-8a6c-dbb6c08c860c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:48.116640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b02bc-9165-4dae-8a6c-dbb6c08c860c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:48.116719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b02bc-9165-4dae-8a6c-dbb6c08c860c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:48.118821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 488/945] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:48.124187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e442-be13-4e06-ba09-2ab38b008b23 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:48.124600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e442-be13-4e06-ba09-2ab38b008b23 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:48.124790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e442-be13-4e06-ba09-2ab38b008b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:48.144000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7359e442-be13-4e06-ba09-2ab38b008b23 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:48.145197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7359e442-be13-4e06-ba09-2ab38b008b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:48.221460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 458/946] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:48.244558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b02bc-9165-4dae-8a6c-dbb6c08c860c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:48.295021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:48.352412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b02bc-9165-4dae-8a6c-dbb6c08c860c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:48.391712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:48.392262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:48.392546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:48.392822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:48.393524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:48.394968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:48.397597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING keystone.server.flask.common [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 07:42:48.437190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.sql.core [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 07:42:48.437190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 07:42:48.437190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [parameters: {'id': 'b38ca875c9d14e4b88c614ee8ada1c71', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 07:42:48.437190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80280) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b38ca875c9d14e4b88c614ee8ada1c71.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b38ca875c9d14e4b88c614ee8ada1c71. Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:42:48.444465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 07:42:48.445256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application yield resource Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise self.value Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application session.commit() Aug 01 07:42:48.446207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.flush() Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application compat.raise_( Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 07:42:48.447803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 07:42:48.448718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:42:48.449540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [parameters: {'id': 'b38ca875c9d14e4b88c614ee8ada1c71', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:48.450418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:48.451214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b38ca875c9d14e4b88c614ee8ada1c71. Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:48.456354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 489/947] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 1 07:42:48 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:48.456925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359e442-be13-4e06-ba09-2ab38b008b23 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:48.558562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359e442-be13-4e06-ba09-2ab38b008b23 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:48.576844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:48.595312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7c951-0b4e-466a-b0c4-d29b77331ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:48.595992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7c951-0b4e-466a-b0c4-d29b77331ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:48.603339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7c951-0b4e-466a-b0c4-d29b77331ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:48.603339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7c951-0b4e-466a-b0c4-d29b77331ef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:48.604209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7c951-0b4e-466a-b0c4-d29b77331ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:48.605664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7c951-0b4e-466a-b0c4-d29b77331ef1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f2b7c951-0b4e-466a-b0c4-d29b77331ef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:48.618115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:48.619690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:48.620608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 459/948] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:48.627286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:48.737076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:48.740436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d9c8e-3eab-4ff3-bd9a-f6e74ae97ed1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:48.777239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3caec-7f5c-43e7-9112-78c81f496b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:48.777866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3caec-7f5c-43e7-9112-78c81f496b00 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:48.778152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3caec-7f5c-43e7-9112-78c81f496b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:48.778426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3caec-7f5c-43e7-9112-78c81f496b00 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:48.779202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3caec-7f5c-43e7-9112-78c81f496b00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:48.779760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3caec-7f5c-43e7-9112-78c81f496b00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:48.781615 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3caec-7f5c-43e7-9112-78c81f496b00 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:48.804752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 490/949] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:49.590723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694d630-880d-467c-a360-df87bbfa351b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.591251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694d630-880d-467c-a360-df87bbfa351b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.591251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694d630-880d-467c-a360-df87bbfa351b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.593002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 460/950] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:49.598472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0e8ce-74e5-4ff6-b87c-b89ed375e8ab None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.598906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0e8ce-74e5-4ff6-b87c-b89ed375e8ab None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.599102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0e8ce-74e5-4ff6-b87c-b89ed375e8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.603730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1664f-9084-445a-8275-12cd9dd2e2a2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.604114 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1664f-9084-445a-8275-12cd9dd2e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.604410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1664f-9084-445a-8275-12cd9dd2e2a2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.606042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 461/951] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:49.612719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eb494-b851-40ed-80f0-968bbd3ab1de None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.613126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eb494-b851-40ed-80f0-968bbd3ab1de None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.613510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eb494-b851-40ed-80f0-968bbd3ab1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.620734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e2f0e8ce-74e5-4ff6-b87c-b89ed375e8ab None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:49.622271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e2f0e8ce-74e5-4ff6-b87c-b89ed375e8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:49.632701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fc1eb494-b851-40ed-80f0-968bbd3ab1de None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:49.634152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fc1eb494-b851-40ed-80f0-968bbd3ab1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:49.702419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 491/952] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:49.703770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583ce10-ffeb-4cde-bf7b-b012f9478c4b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.704070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583ce10-ffeb-4cde-bf7b-b012f9478c4b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.704353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583ce10-ffeb-4cde-bf7b-b012f9478c4b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.705850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 492/953] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:49.713657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ee2b-115b-43ee-9812-1f5079e178b3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.714024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ee2b-115b-43ee-9812-1f5079e178b3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.714424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ee2b-115b-43ee-9812-1f5079e178b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.726486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 462/954] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:49.727404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1eb494-b851-40ed-80f0-968bbd3ab1de None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:49.735659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fe01ee2b-115b-43ee-9812-1f5079e178b3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:49.737447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fe01ee2b-115b-43ee-9812-1f5079e178b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:49.814578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 493/955] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:49.815664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01ee2b-115b-43ee-9812-1f5079e178b3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:49.841377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:49.851973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1eb494-b851-40ed-80f0-968bbd3ab1de None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:49.865066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:49.890998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:49.891690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.892062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.892439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.893074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:49.894459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:49.903599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:49.905103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:49.906035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 463/956] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:49.906992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:49.934141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01ee2b-115b-43ee-9812-1f5079e178b3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:49.972318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:49.972985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.973267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.973538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.984196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:42:49.989162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:49.990126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:49.990994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 494/957] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 07:42:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:49.991741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.022288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3615654-1785-4392-86c7-825fb3a0f4d5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.058967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.060070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.060531 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.060980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.061919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.063730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:50.073835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:50.074998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:50.076055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:50.078418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 464/958] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:50.079327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.079903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.103833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2b61d-0012-4ee9-a8c4-19409743eb3b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.121167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.141190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.141813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.142096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.142375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.143053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:50.143361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.144822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.150087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 495/959] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:50.150698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.196105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971b285-dae4-4a39-a884-f672b21a44ba None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.231524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.232165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.232444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.232713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.233423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:50.233785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.235174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.247567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 465/960] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:50.247567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.265059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1df979-2914-4f82-b115-3601b66981c9 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.301963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.302602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.302888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.303172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.303897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:50.304208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.305590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.312310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 496/961] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:50.313307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.325646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.358126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.362824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366870cc-3bcc-4394-8e06-966c12f80c91 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.401382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.402474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.402944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.403555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.404267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:50.404595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.406045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.419559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 466/962] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:50.419559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.441221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afcd8-841a-4682-a351-3eff0b58381f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.492710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.494883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.495270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.495571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.513057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.515098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:50.537992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:50.539122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:50.540694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:50.540694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:50.540694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:50.540694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:50.540694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 497/963] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:50.541007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466bda-4033-40c4-9375-ec2cd8e3494b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.548604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.583519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.584056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.584307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.584621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.585280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:50.585574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.587231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.597409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 467/964] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:50.598677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.622955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.634634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.681922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23788fc-1f75-4504-8e4a-6c0f276b7625 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.718499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d24365-dc47-4e42-908c-a7c24fd51ecf None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.728383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.729054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.729765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.730047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.731216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:50.731551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.733240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.751409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 498/965] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:50.762101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42113462-94ba-480a-8496-33fd3333ccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.762510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.762929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42113462-94ba-480a-8496-33fd3333ccfe None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.763313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42113462-94ba-480a-8496-33fd3333ccfe None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.763709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42113462-94ba-480a-8496-33fd3333ccfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.764522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42113462-94ba-480a-8496-33fd3333ccfe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.765992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42113462-94ba-480a-8496-33fd3333ccfe None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.871639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 468/966] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:42:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:42:50.880438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f51e8-4ae2-42ad-a653-43258f47f8d2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.910143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c83d-d1d9-45d3-8170-32a7ca2ee51b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.910410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c83d-d1d9-45d3-8170-32a7ca2ee51b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.910700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c83d-d1d9-45d3-8170-32a7ca2ee51b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.911632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.912363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 469/967] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:42:50.918973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555350a1-1c06-46de-8348-773635fe5f7f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.919263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555350a1-1c06-46de-8348-773635fe5f7f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.919625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555350a1-1c06-46de-8348-773635fe5f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.936689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.937643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.938145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.938594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.939822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-555350a1-1c06-46de-8348-773635fe5f7f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:50.941096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-555350a1-1c06-46de-8348-773635fe5f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:50.949029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.950979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:50.967288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:50.968584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:50.969594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 499/968] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:50.974776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:51.031947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 470/969] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:51.050562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555350a1-1c06-46de-8348-773635fe5f7f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:51.066836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:51.109495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fbaa1-d820-43fc-9b01-6884b4c3e096 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:51.147564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:51.148115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:51.148395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:51.148668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:51.149994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:51.150305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:51.151954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:51.173379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 500/970] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 07:42:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:51.179819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:51.195209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555350a1-1c06-46de-8348-773635fe5f7f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:51.219735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:51.245158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:51.246372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:51.246818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:51.247454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:51.249482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:51.251374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:51.255105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING keystone.server.flask.common [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 07:42:51.293228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010919cd-9bc5-4f2f-896e-f7fd80b437df None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:51.306833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.sql.core [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 07:42:51.306833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 07:42:51.306833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [parameters: {'user_id': '5ca3ecbd5c2f4450a5988454368f201f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 07:42:51.306833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80281) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result.read() Aug 01 07:42:51.324203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:42:51.325233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application yield resource Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise self.value Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application session.commit() Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 07:42:51.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.session.flush() Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application compat.raise_( Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 07:42:51.327317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:42:51.334416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result.read() Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:42:51.335325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [parameters: {'user_id': '5ca3ecbd5c2f4450a5988454368f201f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:51.336532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 07:42:51.337421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 471/971] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 07:42:51 2022] POST /identity/v3/users => generated 160 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:51.338454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:51.348318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:51.349032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:51.349319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:51.349592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:51.440527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:51.441901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:51.443412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:51.479609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83a020-4484-49c7-b5c1-af2b344b4fcf None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:51.480003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:51.522650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:51.523798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:51.524163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:51.524974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:51.538251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:51.540797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:51.551891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 501/972] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:42:51 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:42:51.556449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:51.557426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:42:51.558796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:51.560710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:51.562593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 472/973] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:42:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:51.563745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:51.679483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685400e6-d4a6-438b-bea4-0ae6e59dc68f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:51.700811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b0be-8e4f-4193-a0ea-d5d4a52f2939 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:51.719859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:51.720845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:51.721339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:51.721830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:51.724308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:51.734949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:51.737276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:51.744636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:51.745272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:51.745554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:51.745829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:51.746587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:51.746898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:51.749028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:51.750644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:42:51.758030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:51.759418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:51.760354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:42:51.760354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:42:51.760354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:42:51.760354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:51.762669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 502/974] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:51.768983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:51.799593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 473/975] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 07:42:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:51.892260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157336f-4524-49f4-9a29-73065382fc7f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:51.935796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:51.936946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:51.937237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:51.937557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:51.938250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:51.938565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:51.940030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:51.953818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 503/976] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:51.959830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:52.079509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ff95-5a82-473a-873e-834d90375884 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:52.116934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:52.117955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.119092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.119704 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:52.127086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:52.128710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:42:52.141084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:52.142199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:42:52.143338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:52.144877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 474/977] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:52.151215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:52.202325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:52.257491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdab273-3afe-4b67-ac0a-78fb2c4267e8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:52.292622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dafab91-0477-4a80-b1b1-bf44b16340b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:52.293229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafab91-0477-4a80-b1b1-bf44b16340b9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.293556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafab91-0477-4a80-b1b1-bf44b16340b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.293833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafab91-0477-4a80-b1b1-bf44b16340b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:52.294574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dafab91-0477-4a80-b1b1-bf44b16340b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:52.294877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dafab91-0477-4a80-b1b1-bf44b16340b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:52.296379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dafab91-0477-4a80-b1b1-bf44b16340b9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:52.311602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 504/978] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:52.314246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:52.369288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:52.418145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1c7b9-cc70-4907-89e8-ad958703295e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:52.453347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba20cd9-9fe0-448b-bcc9-71ac3cde9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:52.454123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20cd9-9fe0-448b-bcc9-71ac3cde9893 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.454501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20cd9-9fe0-448b-bcc9-71ac3cde9893 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.454869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20cd9-9fe0-448b-bcc9-71ac3cde9893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:52.455783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba20cd9-9fe0-448b-bcc9-71ac3cde9893 None admin] RBAC: Adding query filter params (role.id=1d2b3f7e9adb449db4663a3dba84a1a4, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=3001dbc46cc441cfb61b1d63a4461314) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:52.456192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba20cd9-9fe0-448b-bcc9-71ac3cde9893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:52.457716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba20cd9-9fe0-448b-bcc9-71ac3cde9893 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:52.464248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 475/979] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:42:52 2022] GET /identity/v3/role_assignments?user.id=3001dbc46cc441cfb61b1d63a4461314&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=1d2b3f7e9adb449db4663a3dba84a1a4 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:52.564955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8700a-b1bd-46b1-99fc-858a8261133e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.565330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8700a-b1bd-46b1-99fc-858a8261133e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.565848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8700a-b1bd-46b1-99fc-858a8261133e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:52.567376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 505/980] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:52.573894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a506-734b-477a-be16-af7ff62ae253 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.574180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a506-734b-477a-be16-af7ff62ae253 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.574531 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a506-734b-477a-be16-af7ff62ae253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:52.594214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fea5a506-734b-477a-be16-af7ff62ae253 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:52.595820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fea5a506-734b-477a-be16-af7ff62ae253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:52.667809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 476/981] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:52.690368 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8700a-b1bd-46b1-99fc-858a8261133e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:52.803856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8700a-b1bd-46b1-99fc-858a8261133e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:52.812261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:52.839902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd266b-d98a-4502-aa7f-079543a8155b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:52.840451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd266b-d98a-4502-aa7f-079543a8155b None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.840732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd266b-d98a-4502-aa7f-079543a8155b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.841005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd266b-d98a-4502-aa7f-079543a8155b None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:52.841702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd266b-d98a-4502-aa7f-079543a8155b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:52.843066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd266b-d98a-4502-aa7f-079543a8155b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:52.923573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 506/982] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:52 2022] POST /identity/v3/services => generated 242 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:52.989017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098e1cc-34dd-4508-aac6-3e45f9eb3e7b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.989576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098e1cc-34dd-4508-aac6-3e45f9eb3e7b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.989576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098e1cc-34dd-4508-aac6-3e45f9eb3e7b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:52.991145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 477/983] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:52.996613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc06a8c-49d0-4332-8b56-398805de0dc4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:52.996915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc06a8c-49d0-4332-8b56-398805de0dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:52.997184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc06a8c-49d0-4332-8b56-398805de0dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:53.016974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-dfc06a8c-49d0-4332-8b56-398805de0dc4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:53.018210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dfc06a8c-49d0-4332-8b56-398805de0dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:53.098475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 507/984] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:53.120659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098e1cc-34dd-4508-aac6-3e45f9eb3e7b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:53.173511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:53.229124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098e1cc-34dd-4508-aac6-3e45f9eb3e7b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:53.264704 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:53.265341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:53.265628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:53.265900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:53.266526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:53.267933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:53.277238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:53.278302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:53.279186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:53.280622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 478/985] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:53.284628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc06a8c-49d0-4332-8b56-398805de0dc4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:53.378647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:53.388565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc06a8c-49d0-4332-8b56-398805de0dc4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:53.423641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:53.424337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:53.424638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:53.424911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:53.425590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:53.425898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:53.427279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:53.437367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 508/986] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:53.439058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:53.542010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bcf47-e978-43d5-8026-872fe0cedcd6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:53.552356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:53.577233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:53.578091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:53.578778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:53.579658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:53.580500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:53.580984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:53.582503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:53.592880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 479/987] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:53.594284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:53.704377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0d1c5-cdd7-4b9f-9720-3ef841d059e5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:53.740651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1608e71-73fc-46e4-8923-84add3b65c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:53.741267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1608e71-73fc-46e4-8923-84add3b65c8b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:53.741552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1608e71-73fc-46e4-8923-84add3b65c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:53.741825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1608e71-73fc-46e4-8923-84add3b65c8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:53.742583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1608e71-73fc-46e4-8923-84add3b65c8b None admin] RBAC: Adding query filter params (interface=admin, service_id=71389e780622464bbe8c53b2b9dd3195, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:53.742949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1608e71-73fc-46e4-8923-84add3b65c8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:53.745255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1608e71-73fc-46e4-8923-84add3b65c8b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:53.750558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 509/988] 10.222.0.48 () {56 vars in 1249 bytes} [Mon Aug 1 07:42:53 2022] GET /identity/v3/endpoints?service_id=71389e780622464bbe8c53b2b9dd3195&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:53.756003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:53.864915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afe6c5-b464-4aaa-9cd1-6355bc2f6120 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:53.898374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0bbee-3af5-494e-a636-ff4225d04509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:53.899936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0bbee-3af5-494e-a636-ff4225d04509 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:53.900431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0bbee-3af5-494e-a636-ff4225d04509 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:53.900930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0bbee-3af5-494e-a636-ff4225d04509 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:53.901781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0bbee-3af5-494e-a636-ff4225d04509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:53.903547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0bbee-3af5-494e-a636-ff4225d04509 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:53.910329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 480/989] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:42:53 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:42:54.547854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18d543-a3ec-4fe4-909d-3fd81cf4ae66 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:54.548257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18d543-a3ec-4fe4-909d-3fd81cf4ae66 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:54.548491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18d543-a3ec-4fe4-909d-3fd81cf4ae66 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:54.549840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 510/990] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:54.554824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97308-1221-49a1-99a5-e85950ef1ce6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:54.555292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97308-1221-49a1-99a5-e85950ef1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:54.555705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97308-1221-49a1-99a5-e85950ef1ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:54.574587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c1b97308-1221-49a1-99a5-e85950ef1ce6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:54.575809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c1b97308-1221-49a1-99a5-e85950ef1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:54.606907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:54.645531 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 481/991] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:54.668358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18d543-a3ec-4fe4-909d-3fd81cf4ae66 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:54.691672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:54.779480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18d543-a3ec-4fe4-909d-3fd81cf4ae66 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:54.815672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64682d7b-2615-4469-b564-d26bc3977ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:54.816244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64682d7b-2615-4469-b564-d26bc3977ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:54.816527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64682d7b-2615-4469-b564-d26bc3977ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:54.816809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64682d7b-2615-4469-b564-d26bc3977ca8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:54.817436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64682d7b-2615-4469-b564-d26bc3977ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:54.818807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64682d7b-2615-4469-b564-d26bc3977ca8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-64682d7b-2615-4469-b564-d26bc3977ca8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:54.827938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:54.828959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:54.829833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 511/992] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 07:42:54 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:54.837518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b97308-1221-49a1-99a5-e85950ef1ce6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:54.938290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b97308-1221-49a1-99a5-e85950ef1ce6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:54.951053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ececb-0e6a-410e-be13-25354784df3e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:54.951343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ececb-0e6a-410e-be13-25354784df3e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:54.951651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ececb-0e6a-410e-be13-25354784df3e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:54.953134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 512/993] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:54.957877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aa222-80c6-4ee9-96ca-d949b6df0d49 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:54.958164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aa222-80c6-4ee9-96ca-d949b6df0d49 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:54.958447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aa222-80c6-4ee9-96ca-d949b6df0d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:54.974393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:54.975015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:54.975296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:54.975627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:54.976253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] RBAC: Adding query filter params (name=image) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:54.976557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:54.977042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0f0aa222-80c6-4ee9-96ca-d949b6df0d49 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:54.978457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:54.978745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0f0aa222-80c6-4ee9-96ca-d949b6df0d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:54.988730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 482/994] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:54.989772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:55.022692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:55.052656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 513/995] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:42:55.075572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aa222-80c6-4ee9-96ca-d949b6df0d49 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:55.091044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:55.098584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe8bbe-e679-47fa-8ebd-5c306fc61778 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:55.136836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:55.137800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.138276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.138726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.139463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] RBAC: Adding query filter params (type=image) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:55.139858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:55.141403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:55.146340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 483/996] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:55.179608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aa222-80c6-4ee9-96ca-d949b6df0d49 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:55.218852 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:55.219376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.219692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.219971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.220619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:55.222030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:55.231980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:55.233139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:55.234012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:55.235601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 514/997] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:42:55.239306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:55.347064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:55.353316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757e4ae-7d74-47fb-b0bf-a221d18d939c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:55.390317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:55.390952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.391241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.391519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.392230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] RBAC: Adding query filter params (name=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:55.392540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:55.394007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:55.403560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 484/998] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:55.404695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:55.485543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:55.509965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23bf0-cd17-481d-815e-ac5008439843 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:55.546488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:55.547396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.547884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.548332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.549165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:55.549623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:55.551172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:55.560948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 515/999] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:55.562639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:55.667675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe99135-86f6-4184-bb24-552ef8016cfe None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:55.701333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57597891-0d6e-4bb6-b532-b6acbfb6c570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:55.702009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57597891-0d6e-4bb6-b532-b6acbfb6c570 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.702295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57597891-0d6e-4bb6-b532-b6acbfb6c570 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.702566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57597891-0d6e-4bb6-b532-b6acbfb6c570 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.703315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57597891-0d6e-4bb6-b532-b6acbfb6c570 None admin] RBAC: Adding query filter params (interface=public, service_id=b4653d023012418bac2491594eb552bb, region_id=RegionOne) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:55.703650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57597891-0d6e-4bb6-b532-b6acbfb6c570 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:55.705012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57597891-0d6e-4bb6-b532-b6acbfb6c570 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:55.710279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 485/1000] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:42:55 2022] GET /identity/v3/endpoints?service_id=b4653d023012418bac2491594eb552bb&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:55.716636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:55.823411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40563c7a-03ac-45a1-9d76-ee4ea44b80b4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:55.857775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25536c4b-b3d4-4be7-a47b-1ac71cfb4d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:55.858397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25536c4b-b3d4-4be7-a47b-1ac71cfb4d01 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.858680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25536c4b-b3d4-4be7-a47b-1ac71cfb4d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.858953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25536c4b-b3d4-4be7-a47b-1ac71cfb4d01 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.859627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25536c4b-b3d4-4be7-a47b-1ac71cfb4d01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:55.861007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25536c4b-b3d4-4be7-a47b-1ac71cfb4d01 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:55.867399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 516/1001] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:42:55 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:42:55.938143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbb298-db1a-4d2c-99dd-0ebef61ae3ca None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.938431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbb298-db1a-4d2c-99dd-0ebef61ae3ca None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.938610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbb298-db1a-4d2c-99dd-0ebef61ae3ca None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.940188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 486/1002] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:55.945574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba05cf3-0ab0-46a2-b899-eecb78e1e4b9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:55.946458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba05cf3-0ab0-46a2-b899-eecb78e1e4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:55.946851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba05cf3-0ab0-46a2-b899-eecb78e1e4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:55.960840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:55.968492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1ba05cf3-0ab0-46a2-b899-eecb78e1e4b9 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:55.969958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1ba05cf3-0ab0-46a2-b899-eecb78e1e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:56.046740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 517/1003] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:56.069964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbb298-db1a-4d2c-99dd-0ebef61ae3ca None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:56.096456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:56.177058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbb298-db1a-4d2c-99dd-0ebef61ae3ca None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:56.214717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:56.215272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:56.215571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:56.215851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:56.216475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:56.217848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:56.227241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:56.228486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:42:56.229357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:42:56.230804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 487/1004] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 07:42:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:56.234947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba05cf3-0ab0-46a2-b899-eecb78e1e4b9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:56.342769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba05cf3-0ab0-46a2-b899-eecb78e1e4b9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:56.377464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:56.378081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:56.378362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:56.378646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:56.379306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:56.379633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:56.381006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:56.382789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:56.387580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 518/1005] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:56.393585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:56.484200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:56.497120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a25f80-4bf5-4509-a4bf-35362362ad67 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:56.532094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c40dfa-cdb2-44ca-8f4d-30c0ec7a1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:56.532714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40dfa-cdb2-44ca-8f4d-30c0ec7a1fbf None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:56.533001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40dfa-cdb2-44ca-8f4d-30c0ec7a1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:56.533278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40dfa-cdb2-44ca-8f4d-30c0ec7a1fbf None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:56.533956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c40dfa-cdb2-44ca-8f4d-30c0ec7a1fbf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:56.534263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c40dfa-cdb2-44ca-8f4d-30c0ec7a1fbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:56.535661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c40dfa-cdb2-44ca-8f4d-30c0ec7a1fbf None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:56.541169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 488/1006] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 07:42:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:56.546676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:56.650047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3967624-eec4-4879-aae0-9cf427bbedc1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:56.684129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ab83d-b8e9-44c6-91ee-0b6baa440676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:56.684759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ab83d-b8e9-44c6-91ee-0b6baa440676 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:56.685046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ab83d-b8e9-44c6-91ee-0b6baa440676 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:56.685323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ab83d-b8e9-44c6-91ee-0b6baa440676 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:56.686043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ab83d-b8e9-44c6-91ee-0b6baa440676 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:56.687424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ab83d-b8e9-44c6-91ee-0b6baa440676 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:56.771553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 519/1007] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:42:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:57.164596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce98d1-8a14-4931-b42b-6f986cfc9a3a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:57.165726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce98d1-8a14-4931-b42b-6f986cfc9a3a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:57.165726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce98d1-8a14-4931-b42b-6f986cfc9a3a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:57.167029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 489/1008] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:57.172225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5040-bcea-4515-90c9-3603b52738a3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:57.172613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5040-bcea-4515-90c9-3603b52738a3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:57.174970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5040-bcea-4515-90c9-3603b52738a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:57.191275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-638e5040-bcea-4515-90c9-3603b52738a3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:57.192500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-638e5040-bcea-4515-90c9-3603b52738a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:57.251866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:57.269115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 520/1009] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:57.291408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce98d1-8a14-4931-b42b-6f986cfc9a3a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:57.393524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce98d1-8a14-4931-b42b-6f986cfc9a3a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:57.428488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01f6eb-7656-4bb7-a587-2d8aa16b5bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:57.429040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01f6eb-7656-4bb7-a587-2d8aa16b5bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:57.429324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01f6eb-7656-4bb7-a587-2d8aa16b5bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:57.429597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01f6eb-7656-4bb7-a587-2d8aa16b5bd3 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:57.430853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01f6eb-7656-4bb7-a587-2d8aa16b5bd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:57.432267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01f6eb-7656-4bb7-a587-2d8aa16b5bd3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:57.501378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 490/1010] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:57 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:57.893551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd51255-5f6f-4fad-b7fe-9b6d86918095 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:57.893874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd51255-5f6f-4fad-b7fe-9b6d86918095 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:57.894120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd51255-5f6f-4fad-b7fe-9b6d86918095 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:57.895677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 521/1011] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:57.900633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165210ce-7f69-42af-9253-5ce12df174cc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:57.900945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165210ce-7f69-42af-9253-5ce12df174cc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:57.901221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165210ce-7f69-42af-9253-5ce12df174cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:57.920031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-165210ce-7f69-42af-9253-5ce12df174cc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:57.921237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-165210ce-7f69-42af-9253-5ce12df174cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:57.929498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:58.001181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 491/1012] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:57 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:58.023237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd51255-5f6f-4fad-b7fe-9b6d86918095 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:58.125578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd51255-5f6f-4fad-b7fe-9b6d86918095 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:58.159575 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:58.160223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:58.160507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:58.160785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:58.161502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:58.162890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:58.172706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:58.175803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:58.176741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 522/1013] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:42:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:58.179926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165210ce-7f69-42af-9253-5ce12df174cc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:58.282183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165210ce-7f69-42af-9253-5ce12df174cc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:58.315615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:58.316237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:58.316522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:58.316796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:58.317466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:58.317771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:58.319138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:58.324725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 492/1014] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:58.330362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:58.381257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:58.433289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86855bdb-b4f2-4a5f-a714-a3fbc52344ea None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:58.468634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b96a9c-1f0a-41d8-b3b1-b5f49a7df8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:58.469264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b96a9c-1f0a-41d8-b3b1-b5f49a7df8cb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:58.469551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b96a9c-1f0a-41d8-b3b1-b5f49a7df8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:58.469828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b96a9c-1f0a-41d8-b3b1-b5f49a7df8cb None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:58.470513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b96a9c-1f0a-41d8-b3b1-b5f49a7df8cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:58.470819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b96a9c-1f0a-41d8-b3b1-b5f49a7df8cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:58.472229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b96a9c-1f0a-41d8-b3b1-b5f49a7df8cb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:58.477439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 523/1015] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 07:42:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:58.482863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:58.485737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:58.586182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c0827-3acd-401a-b516-e9d411599294 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:58.619928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d06f9b-d25f-4fe0-ab28-26251394905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:58.620582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d06f9b-d25f-4fe0-ab28-26251394905a None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:58.620866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d06f9b-d25f-4fe0-ab28-26251394905a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:58.621144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d06f9b-d25f-4fe0-ab28-26251394905a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:58.621890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d06f9b-d25f-4fe0-ab28-26251394905a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:58.623320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d06f9b-d25f-4fe0-ab28-26251394905a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:58.694971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 493/1016] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:42:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:42:58.775741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50bc6-41ea-4272-95b3-80388c4685fe None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:58.776058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50bc6-41ea-4272-95b3-80388c4685fe None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:58.776303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50bc6-41ea-4272-95b3-80388c4685fe None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:58.777957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 524/1017] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:58.783371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f1df1-99e7-406c-81aa-7243e870d7d6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:58.783690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f1df1-99e7-406c-81aa-7243e870d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:58.783968 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f1df1-99e7-406c-81aa-7243e870d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:58.803218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c73f1df1-99e7-406c-81aa-7243e870d7d6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:58.804527 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c73f1df1-99e7-406c-81aa-7243e870d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:58.866812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:58.887897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 494/1018] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:58.910152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c50bc6-41ea-4272-95b3-80388c4685fe None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:59.020827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c50bc6-41ea-4272-95b3-80388c4685fe None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:59.036426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:59.058165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345fae-219a-4285-b7c7-68e1e4fcbc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:59.058825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345fae-219a-4285-b7c7-68e1e4fcbc91 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:59.059116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345fae-219a-4285-b7c7-68e1e4fcbc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:59.059393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345fae-219a-4285-b7c7-68e1e4fcbc91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:59.060594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345fae-219a-4285-b7c7-68e1e4fcbc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:59.062150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345fae-219a-4285-b7c7-68e1e4fcbc91 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:59.064029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 525/1019] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:42:59.069734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f1df1-99e7-406c-81aa-7243e870d7d6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:59.172726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f1df1-99e7-406c-81aa-7243e870d7d6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:59.206686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce1f0-527b-4c82-a0c2-a888427f7169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:59.207339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce1f0-527b-4c82-a0c2-a888427f7169 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:59.207684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce1f0-527b-4c82-a0c2-a888427f7169 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:59.207915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce1f0-527b-4c82-a0c2-a888427f7169 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:59.208615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbce1f0-527b-4c82-a0c2-a888427f7169 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:59.210048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbce1f0-527b-4c82-a0c2-a888427f7169 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:59.291073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 495/1020] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:42:59 2022] POST /identity/v3/projects => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:42:59.546518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f3cde-eac7-4728-9a00-e71d1fa96306 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:59.547011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f3cde-eac7-4728-9a00-e71d1fa96306 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:59.547011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f3cde-eac7-4728-9a00-e71d1fa96306 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:59.548524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 526/1021] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:59.553659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58060b0a-1e6c-4a35-bfd7-c56de610c0bf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:59.554020 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58060b0a-1e6c-4a35-bfd7-c56de610c0bf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:59.554366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58060b0a-1e6c-4a35-bfd7-c56de610c0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:59.573637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-58060b0a-1e6c-4a35-bfd7-c56de610c0bf None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:59.574822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-58060b0a-1e6c-4a35-bfd7-c56de610c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:59.605947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:59.645373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 496/1022] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:42:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:42:59.668479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f3cde-eac7-4728-9a00-e71d1fa96306 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:59.778862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f3cde-eac7-4728-9a00-e71d1fa96306 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:59.815429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:59.816025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:59.816314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:59.816592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] PATH_INFO: `/v3/services/image` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:59.817224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:59.818603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:42:59.828433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:42:59.829544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:42:59.835978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 527/1023] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 07:42:59 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:42:59.837294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58060b0a-1e6c-4a35-bfd7-c56de610c0bf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:59.940669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58060b0a-1e6c-4a35-bfd7-c56de610c0bf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:59.976121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:59.976745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:59.977598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:59.977871 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:59.978548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] RBAC: Adding query filter params (name=image) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:42:59.978849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:59.980236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:59.989772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 497/1024] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:42:59.990855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:00.040595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:00.094908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de2ad4-4119-48b1-a2f3-54c25260a317 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:00.128783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:00.129406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:00.129694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:00.129972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:00.130653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] RBAC: Adding query filter params (type=image) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:00.130963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:00.132373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:00.137460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 528/1025] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:42:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:00.143079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:00.175652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:00.305163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03164389-e451-46d8-93f3-17ad82a4206b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:00.340382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0ce46-1796-4727-8764-097096757d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:00.341005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ce46-1796-4727-8764-097096757d8a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:00.341289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ce46-1796-4727-8764-097096757d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:00.341562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ce46-1796-4727-8764-097096757d8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:00.342308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0ce46-1796-4727-8764-097096757d8a None admin] RBAC: Adding query filter params (interface=public, service_id=bdc55b677aac4fc6814897fd617ccb05, region_id=RegionOne) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:00.342617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0ce46-1796-4727-8764-097096757d8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:00.344011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0ce46-1796-4727-8764-097096757d8a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:00.353776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 498/1026] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:43:00 2022] GET /identity/v3/endpoints?service_id=bdc55b677aac4fc6814897fd617ccb05&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:00.354995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:00.459698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4f7b-0d45-4017-bfca-92b63ef7f8eb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:00.471915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:00.495041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fb09e-778d-4bc7-a868-d3b8e5bcb739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:00.495695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb09e-778d-4bc7-a868-d3b8e5bcb739 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:00.495977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb09e-778d-4bc7-a868-d3b8e5bcb739 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:00.496246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb09e-778d-4bc7-a868-d3b8e5bcb739 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:00.496896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fb09e-778d-4bc7-a868-d3b8e5bcb739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:00.498267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fb09e-778d-4bc7-a868-d3b8e5bcb739 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:00.505477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 529/1027] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:43:00 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:00.847613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae443-f12d-4a92-8dbb-be0704f13263 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:00.848015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae443-f12d-4a92-8dbb-be0704f13263 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:00.848304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae443-f12d-4a92-8dbb-be0704f13263 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:00.849705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 499/1028] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:00.854875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb47ad-5882-4a77-aeab-ae13598740ea None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:00.855354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb47ad-5882-4a77-aeab-ae13598740ea None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:00.855434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb47ad-5882-4a77-aeab-ae13598740ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:00.874155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-96eb47ad-5882-4a77-aeab-ae13598740ea None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:00.875349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96eb47ad-5882-4a77-aeab-ae13598740ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:00.947646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 530/1029] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:00.966468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae443-f12d-4a92-8dbb-be0704f13263 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.062745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:01.070725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae443-f12d-4a92-8dbb-be0704f13263 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.104924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.105560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.105840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.106108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.116299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:01.121780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:01.122764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:01.123696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 500/1030] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 07:43:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:01.128979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb47ad-5882-4a77-aeab-ae13598740ea None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.230711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb47ad-5882-4a77-aeab-ae13598740ea None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.233157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:01.267376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc04abb-3770-45d6-9eb0-9eb6c0feb170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.268028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc04abb-3770-45d6-9eb0-9eb6c0feb170 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.268319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc04abb-3770-45d6-9eb0-9eb6c0feb170 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.268599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc04abb-3770-45d6-9eb0-9eb6c0feb170 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.269320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc04abb-3770-45d6-9eb0-9eb6c0feb170 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:01.269629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc04abb-3770-45d6-9eb0-9eb6c0feb170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:01.271019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc04abb-3770-45d6-9eb0-9eb6c0feb170 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:01.280105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 531/1031] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 07:43:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:01.285547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.365884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35924-15f7-44f7-b13d-70afe6071f1b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.366195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35924-15f7-44f7-b13d-70afe6071f1b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.366458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35924-15f7-44f7-b13d-70afe6071f1b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.368508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 532/1032] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:01.373271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780222a4-2791-43ef-8e1b-de4919cd96d7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.373567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780222a4-2791-43ef-8e1b-de4919cd96d7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.373854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780222a4-2791-43ef-8e1b-de4919cd96d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.392358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-780222a4-2791-43ef-8e1b-de4919cd96d7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:01.392609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c358464-030b-4722-b9e5-725121ca7755 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.393554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-780222a4-2791-43ef-8e1b-de4919cd96d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:01.431736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.432659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.433281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.433569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.444622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:01.446918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:01.463567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 533/1033] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:01.463725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:01.464593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:01.465841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:43:01.465841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:01.465841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:01.465841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:01.468901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 501/1034] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:01.469891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780222a4-2791-43ef-8e1b-de4919cd96d7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.487467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.500921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:01.573168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:01.581040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780222a4-2791-43ef-8e1b-de4919cd96d7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.602758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc15f7-e816-455b-9dd4-2d43428ef38f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.621268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.621830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.622298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.622407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.623108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:01.623425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:01.624914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:01.642561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 534/1035] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:01.643424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.643731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.644170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.644460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.644740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.645902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:01.647360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:01.649539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 502/1036] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:01.655089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.740174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:01.751680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fde3e-d989-4e3c-957b-8fca6ea07fda None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.771336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb99c4-0662-477f-8ddd-669fd5377347 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.793606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.794670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.794964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.795246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.803246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:01.804756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:01.809795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.810634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.811112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.811629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.812501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:01.814106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:43:01.816489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:01.817458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 535/1037] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:43:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:01.823855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:01.843784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:01.901001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 503/1038] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:43:01 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:43:01.940191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f913fa-84ea-4aaf-afc4-3b126b3dd170 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:01.980991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:01.981640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:01.981929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:01.982208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:01.983031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:01.983346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:01.984876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:02.004796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 536/1039] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:43:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:02.011326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:02.125899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281a991-fe99-442e-a370-58606dbe6718 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:02.156941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:02.162700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:02.163394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.163683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.169341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.233944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:02.235401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:02.334812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 504/1040] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:43:02.341132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:02.417138 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:02.444192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344394e4-4fd3-4b60-92f6-9331cddad479 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:02.479490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9edf4-2600-4672-b012-c3c60f00cfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:02.480165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9edf4-2600-4672-b012-c3c60f00cfde None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.480447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9edf4-2600-4672-b012-c3c60f00cfde None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.480713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9edf4-2600-4672-b012-c3c60f00cfde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.490902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9edf4-2600-4672-b012-c3c60f00cfde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:02.492939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9edf4-2600-4672-b012-c3c60f00cfde None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-5ca9edf4-2600-4672-b012-c3c60f00cfde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:02.507644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:02.508964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:02.509855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:43:02.509855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:02.509855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:02.509855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:02.509855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 537/1041] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:02.516166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:02.525744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fae3a-8313-4048-bce1-46e1a763be85 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.526032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fae3a-8313-4048-bce1-46e1a763be85 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.526310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fae3a-8313-4048-bce1-46e1a763be85 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.527743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 538/1042] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:43:02.532638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5500013-e46d-4762-affe-5eed4f3ee4b6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.532929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5500013-e46d-4762-affe-5eed4f3ee4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.533206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5500013-e46d-4762-affe-5eed4f3ee4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.551628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:02.552064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a5500013-e46d-4762-affe-5eed4f3ee4b6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:02.553267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a5500013-e46d-4762-affe-5eed4f3ee4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:02.627778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 539/1043] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:02.631334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06282c6d-0076-410e-89e1-2fe454727ee5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:02.650113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5500013-e46d-4762-affe-5eed4f3ee4b6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:02.668707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:02.669639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.670144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.670641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.671617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:02.672174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:02.673684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:02.685995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 505/1044] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:02.691313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:02.709556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:02.763576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5500013-e46d-4762-affe-5eed4f3ee4b6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:02.790673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:02.800287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:02.800668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477942-27e6-4687-872f-57eefc3fe03c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:02.804318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.804318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.804318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] PATH_INFO: `/v3/services/image` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.804318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:02.804318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:02.816077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:02.817108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:02.817995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 540/1045] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:02.824207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:02.842708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:02.843388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.843706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.844069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.851302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:02.853344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:43:02.864939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:02.865831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:02.866928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 506/1046] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:02.872095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:02.934282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e145286-1581-4bc8-8f72-2d8f3fcc4d3c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:02.955181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:02.970853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:02.972436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:02.972436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:02.972436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:02.972667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] RBAC: Adding query filter params (name=image) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:02.972980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:02.974352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:02.979723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998e828-6f56-44fb-987a-ebeaf5e8e2b4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:02.980570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 541/1047] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:02.986045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:03.015661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:03.016288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:03.016573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:03.016851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:03.017625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:03.017939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:03.019383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:03.031160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 507/1048] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:03.036867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:03.049384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:03.095037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4892a-0b34-4c7b-9267-93dd57467f95 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:03.129921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:03.130541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:03.130823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:03.131094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:03.131794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] RBAC: Adding query filter params (type=image) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:03.132097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:03.134413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:03.139570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 542/1049] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:03.145092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:03.150568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd45b8e-cda4-4962-881d-145c4ec18e76 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:03.188301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbd7de-6df1-4f8c-b85f-36bb6d7d62ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:03.188853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbd7de-6df1-4f8c-b85f-36bb6d7d62ce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:03.189194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbd7de-6df1-4f8c-b85f-36bb6d7d62ce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:03.189477 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbd7de-6df1-4f8c-b85f-36bb6d7d62ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:03.190266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbd7de-6df1-4f8c-b85f-36bb6d7d62ce None admin] RBAC: Adding query filter params (role.id=1d2b3f7e9adb449db4663a3dba84a1a4, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=659b367f1ce643a68d55ff1c00d8b7b8) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:03.190584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbd7de-6df1-4f8c-b85f-36bb6d7d62ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:03.192053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbd7de-6df1-4f8c-b85f-36bb6d7d62ce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:03.198412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 508/1050] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:03 2022] GET /identity/v3/role_assignments?user.id=659b367f1ce643a68d55ff1c00d8b7b8&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=1d2b3f7e9adb449db4663a3dba84a1a4 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:03.199235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:03.259752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0d7aa-799c-45d3-86a5-8a52c3fbdc82 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:03.297461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5077c88-1b5c-42f4-b0ba-f6a567fdd07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:03.298153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5077c88-1b5c-42f4-b0ba-f6a567fdd07f None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:03.298441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5077c88-1b5c-42f4-b0ba-f6a567fdd07f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:03.298716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5077c88-1b5c-42f4-b0ba-f6a567fdd07f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:03.299430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5077c88-1b5c-42f4-b0ba-f6a567fdd07f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:03.300862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5077c88-1b5c-42f4-b0ba-f6a567fdd07f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:03.380806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 543/1051] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:43:03.989209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1233be7-d385-4756-be49-fb7646f3d12f None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:03.989725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1233be7-d385-4756-be49-fb7646f3d12f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:03.989804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1233be7-d385-4756-be49-fb7646f3d12f None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:03.991254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 509/1052] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:43:03.996425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c39b7c-b504-4754-be4a-18a20eebeaf9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:03.996820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c39b7c-b504-4754-be4a-18a20eebeaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:03.996991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c39b7c-b504-4754-be4a-18a20eebeaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:04.015951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-10c39b7c-b504-4754-be4a-18a20eebeaf9 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:04.017083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-10c39b7c-b504-4754-be4a-18a20eebeaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:04.096867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 544/1053] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:04.120434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1233be7-d385-4756-be49-fb7646f3d12f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:04.198776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:04.225240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1233be7-d385-4756-be49-fb7646f3d12f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:04.266787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:04.267349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:04.267658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:04.267936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:04.278804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:04.283718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:04.285213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:04.286469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:04.293703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 510/1054] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:04.294003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c39b7c-b504-4754-be4a-18a20eebeaf9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:04.323962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:04.396279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c39b7c-b504-4754-be4a-18a20eebeaf9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:04.431868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:04.432489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:04.432771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:04.433044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:04.433756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:04.434064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:04.435444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:04.443739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 545/1055] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:04.449406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:04.556996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980f865-4d6f-43ff-a5ec-f0895ba05f06 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:04.591072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:04.591748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:04.592035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:04.592312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] PATH_INFO: `/v3/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:04.594234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:04.607451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] RBAC: Authorizing `identity:get_user(user_id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:04.608927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:04.615313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 511/1056] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:04 2022] GET /identity/v3/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:04.616478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:04.708115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:04.717940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e406a-3fb6-4422-811d-bf7e7a1195f3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:04.752046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:04.752676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:04.752966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:04.753247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:04.827010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:04.828733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:04.921545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:04.933222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 546/1057] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:04.938129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.045475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedb095-5724-4630-b66a-74a8e7ca3e13 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:05.081671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b3989-038d-42b6-8dc7-fce14e4360a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:05.082240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3989-038d-42b6-8dc7-fce14e4360a6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.082525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3989-038d-42b6-8dc7-fce14e4360a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.082803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3989-038d-42b6-8dc7-fce14e4360a6 None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.083985 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b3989-038d-42b6-8dc7-fce14e4360a6 None admin] RBAC: Authorizing `identity:get_project(project_id=18945f05f1b24a21b755146f1f3b565c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:05.085693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b3989-038d-42b6-8dc7-fce14e4360a6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:05.088206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 512/1058] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:04 2022] GET /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:05.094120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.197901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98c09-abb1-436b-b944-ca00e735866e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:05.210992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00fe48-4661-4f91-9d24-cb28cb0b29f1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.211334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00fe48-4661-4f91-9d24-cb28cb0b29f1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.211640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00fe48-4661-4f91-9d24-cb28cb0b29f1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.213173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 513/1059] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:05.217931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f4ee7-c8d3-45b9-b850-c78c375a5091 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.218228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f4ee7-c8d3-45b9-b850-c78c375a5091 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.218511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f4ee7-c8d3-45b9-b850-c78c375a5091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.234656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:05.235309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.235660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.235946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.236736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=18945f05f1b24a21b755146f1f3b565c, user.id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:05.237048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:05.237900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9e8f4ee7-c8d3-45b9-b850-c78c375a5091 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:05.238484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:05.239098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9e8f4ee7-c8d3-45b9-b850-c78c375a5091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:05.251572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 547/1060] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:05 2022] GET /identity/v3/role_assignments?user.id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3&scope.project.id=18945f05f1b24a21b755146f1f3b565c&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:05.307130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:05.318734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 514/1061] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:43:05.342793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.405805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:05.421197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf12295-42fc-4add-aa5c-ebbd44a95247 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.421486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf12295-42fc-4add-aa5c-ebbd44a95247 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.421763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf12295-42fc-4add-aa5c-ebbd44a95247 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.423222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 515/1062] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:43:05.428498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca6d7f-f184-4432-a4c0-f1dcf05e1768 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.428790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca6d7f-f184-4432-a4c0-f1dcf05e1768 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.429097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca6d7f-f184-4432-a4c0-f1dcf05e1768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.449843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3dca6d7f-f184-4432-a4c0-f1dcf05e1768 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:05.451077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3dca6d7f-f184-4432-a4c0-f1dcf05e1768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:05.452928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c362-e775-49ac-bcc6-ce83fecb0fdd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:05.490643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:05.491294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.491610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.491894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.503101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:05.508740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:05.509885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:05.510884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 548/1063] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 07:43:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:05.515717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.531419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 516/1064] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:43:05.547937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca6d7f-f184-4432-a4c0-f1dcf05e1768 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.626191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083bb33-7f03-4815-a010-c2ece7028354 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:05.642839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:05.654972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca6d7f-f184-4432-a4c0-f1dcf05e1768 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:05.660929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:05.661561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.661910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.662127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.662856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:05.663168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:05.665145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:05.673933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 549/1065] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 07:43:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:05.680530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.683174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:05.697929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:05.698637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.698926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.699224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.775957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:05.777422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:05.797565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dd8d8-c1ce-4673-b241-be27e9942152 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:05.835267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:05.836659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:05.836949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:05.837228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:05.847885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:05.849356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:05.864938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:05.866058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:05.867863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:43:05.867863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:05.867863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:05.867863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:05.871276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 550/1066] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:05.872130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.899825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 517/1067] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:43:05.901236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:05.905567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:05.923519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:05.985187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bee89-dfa9-48c4-a83b-bc748caf7b4a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.011760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aed3f-98c4-421c-a3b9-8e95687442b0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.024930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.025559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.025850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.026121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.026824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:06.027137 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.028622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.045993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 551/1068] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:06.047000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.051419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.052102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.052383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.052654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.068259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.069720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:06.086735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:06.087719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:06.088649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:43:06.088649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:06.088649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:06.088649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:06.088649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 518/1069] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:06.095181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.146990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:06.157839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5cc3b-4210-4826-8895-776613a6fa25 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.171557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:06.193556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.194204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.194495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.194775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.202872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.204362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.205508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf536dc-1984-4488-83ed-390c2759e417 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:43:06.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:06.217975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:06.219071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 552/1070] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:06.223771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.247796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.248431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.248719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.248994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.249720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:06.250034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.251402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.264940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 519/1071] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:06.339985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1449-0e86-4de8-b63d-f0ec23ff3216 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.378498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.379164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.379454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.379768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.380620 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:06.380936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.382977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.404025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 553/1072] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:06.412936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.521669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52417b-a77c-4616-b8c2-7a142c72c3c2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.553466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:06.558832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.559484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.559807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.560084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.571027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.573158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:06.588185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:06.589258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:06.590358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:43:06.590358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:06.590358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:06.590358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:06.590358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 520/1073] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:06.596618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.607404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:06.698819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20cd5-a2c6-418c-8eb5-c2d1e7a02113 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.731386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.732038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.732322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.732596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.733274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:06.733576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.734939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.746291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 554/1074] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:06.752318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.851756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18451a-4e4a-44fb-b319-150b76ffda69 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.885161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.885804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.886083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.886353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.892837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.894274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:43:06.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:06.906957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:06.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 521/1075] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:06.912935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.981291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:07.014910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2fa99-7652-4a1c-80bc-12172abea6a6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:07.048578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb01d98-7421-40a4-8af4-43896c1bb2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:07.049257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb01d98-7421-40a4-8af4-43896c1bb2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:07.049542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb01d98-7421-40a4-8af4-43896c1bb2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:07.049816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb01d98-7421-40a4-8af4-43896c1bb2d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:07.050561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb01d98-7421-40a4-8af4-43896c1bb2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:07.050867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb01d98-7421-40a4-8af4-43896c1bb2d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:07.052339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb01d98-7421-40a4-8af4-43896c1bb2d2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:07.064669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 555/1076] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:07.071075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:07.116671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:07.171683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25765fe7-ac3f-410e-a46a-b5904d5e9d43 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:07.204859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71686962-0bc0-4aea-ab4e-e783455a01a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:07.205521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71686962-0bc0-4aea-ab4e-e783455a01a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:07.205802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71686962-0bc0-4aea-ab4e-e783455a01a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:07.206073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71686962-0bc0-4aea-ab4e-e783455a01a4 None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/659b367f1ce643a68d55ff1c00d8b7b8/roles/1d2b3f7e9adb449db4663a3dba84a1a4` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:07.208410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71686962-0bc0-4aea-ab4e-e783455a01a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=659b367f1ce643a68d55ff1c00d8b7b8, role_id=1d2b3f7e9adb449db4663a3dba84a1a4)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:07.210145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71686962-0bc0-4aea-ab4e-e783455a01a4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:07.265519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 522/1077] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:43:07 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/659b367f1ce643a68d55ff1c00d8b7b8/roles/1d2b3f7e9adb449db4663a3dba84a1a4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:07.285354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f03b-7513-4d97-93e1-c6d5683d68af None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:07.285893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f03b-7513-4d97-93e1-c6d5683d68af None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:07.285893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f03b-7513-4d97-93e1-c6d5683d68af None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:07.287395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 556/1078] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:07.293657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72812c0-96bb-473c-9ebd-e4632d61305d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:07.293949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72812c0-96bb-473c-9ebd-e4632d61305d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:07.294229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72812c0-96bb-473c-9ebd-e4632d61305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:07.313567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b72812c0-96bb-473c-9ebd-e4632d61305d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:07.314823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b72812c0-96bb-473c-9ebd-e4632d61305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:07.388474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 523/1079] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:07.411442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922f03b-7513-4d97-93e1-c6d5683d68af None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:07.514090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922f03b-7513-4d97-93e1-c6d5683d68af None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:07.540366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:07.548178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:07.548824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:07.549337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:07.549465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:07.560343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:07.565326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:07.566443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:07.572987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 557/1080] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:07.574931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72812c0-96bb-473c-9ebd-e4632d61305d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:07.609267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:07.675515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72812c0-96bb-473c-9ebd-e4632d61305d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:07.709262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:07.710133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:07.710418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:07.710719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:07.711444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:07.711779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:07.713146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:07.721512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 524/1081] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:07.727780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:07.827334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf90939-4947-44ce-8cac-799ac5bc69e3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:07.861344 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:07.861969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:07.862251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:07.862523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] PATH_INFO: `/v3/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:07.863726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] RBAC: Authorizing `identity:get_user(user_id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:07.865159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:07.871922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 558/1082] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:07 2022] GET /identity/v3/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:07.873102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:07.967160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:07.972841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4781cf-2a04-43fb-98c8-f18ae1e74c56 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:08.006337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0340f-92c6-4c6d-b5f0-f7eb072c6ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:08.006977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0340f-92c6-4c6d-b5f0-f7eb072c6ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:08.007260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0340f-92c6-4c6d-b5f0-f7eb072c6ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:08.007584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0340f-92c6-4c6d-b5f0-f7eb072c6ea0 None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:08.008735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0340f-92c6-4c6d-b5f0-f7eb072c6ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=18945f05f1b24a21b755146f1f3b565c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:08.010166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0340f-92c6-4c6d-b5f0-f7eb072c6ea0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:08.011878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 525/1083] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:07 2022] GET /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:08.017845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:08.074330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:08.118136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a4fcd-2330-4ad7-bdc6-344c5649c9ca None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:08.151593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc9360-2ed3-43e2-b344-69a4122d377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:08.152255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc9360-2ed3-43e2-b344-69a4122d377e None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:08.152539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc9360-2ed3-43e2-b344-69a4122d377e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:08.152816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc9360-2ed3-43e2-b344-69a4122d377e None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:08.154751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc9360-2ed3-43e2-b344-69a4122d377e None admin] RBAC: Authorizing `identity:create_grant(project_id=18945f05f1b24a21b755146f1f3b565c, user_id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:08.156226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc9360-2ed3-43e2-b344-69a4122d377e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:08.224404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 559/1084] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:43:08 2022] PUT /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:43:09.268849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628bf8b-7033-4fc1-8b83-2a66244f648c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:09.269370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628bf8b-7033-4fc1-8b83-2a66244f648c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:09.269370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628bf8b-7033-4fc1-8b83-2a66244f648c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:09.270743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 526/1085] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:09.276748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4350-8f97-43a8-b6c6-1b37ca658ac5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:09.277159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4350-8f97-43a8-b6c6-1b37ca658ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:09.277295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4350-8f97-43a8-b6c6-1b37ca658ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:09.295295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e8bd4350-8f97-43a8-b6c6-1b37ca658ac5 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:09.296558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e8bd4350-8f97-43a8-b6c6-1b37ca658ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:09.364244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 560/1086] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:09.387038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628bf8b-7033-4fc1-8b83-2a66244f648c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:09.485663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628bf8b-7033-4fc1-8b83-2a66244f648c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:09.517984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:09.518608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:09.518939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:09.519155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:09.529529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:09.534370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:09.535356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:09.536829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 527/1087] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 07:43:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:09.541503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd4350-8f97-43a8-b6c6-1b37ca658ac5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:09.586452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:09.640861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd4350-8f97-43a8-b6c6-1b37ca658ac5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:09.674475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:09.675091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:09.675371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:09.675701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:09.676415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:09.676715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:09.678090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:09.685876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 561/1088] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 07:43:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:09.691598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:09.717622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:09.793444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872012c2-7e8c-48ed-91b8-651cb96611df None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:09.826485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:09.827128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:09.827414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:09.827725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:09.838684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:09.840169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:09.855260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:09.856174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:09.857089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:43:09.857089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:09.857089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:09.857089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:09.857089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 528/1089] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:09.862329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:09.961724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bce5b-8808-46a9-a15a-90fa28746031 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:09.970382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:09.995765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:09.996381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:09.996665 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:09.996938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:09.997621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:09.997926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:09.999301 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.010609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 562/1090] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:10.016251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.093591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:10.116069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa303e-f18a-458d-998b-f6ec9d04397a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.151455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e27ef-ea6a-4a57-8fb6-caff8ffca99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.152121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e27ef-ea6a-4a57-8fb6-caff8ffca99d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.152406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e27ef-ea6a-4a57-8fb6-caff8ffca99d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.152684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e27ef-ea6a-4a57-8fb6-caff8ffca99d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.160157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e27ef-ea6a-4a57-8fb6-caff8ffca99d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.161623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e27ef-ea6a-4a57-8fb6-caff8ffca99d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7d5e27ef-ea6a-4a57-8fb6-caff8ffca99d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:43:10.173351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:10.174358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:43:10.175392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:10.181030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 529/1091] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:10.181105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.199353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f8550-2123-47be-b1fe-2aff634d8078 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.199688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f8550-2123-47be-b1fe-2aff634d8078 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.199971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f8550-2123-47be-b1fe-2aff634d8078 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.201344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 530/1092] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:43:10.206352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b373fac-ad96-49ac-978e-eea73795c029 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.206352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b373fac-ad96-49ac-978e-eea73795c029 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.206622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b373fac-ad96-49ac-978e-eea73795c029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.224662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7b373fac-ad96-49ac-978e-eea73795c029 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:10.225848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7b373fac-ad96-49ac-978e-eea73795c029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:10.280850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158cde8-111f-4cb9-bc42-68582e6aa637 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.292199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 531/1093] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:43:10.315815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b373fac-ad96-49ac-978e-eea73795c029 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.316066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.316688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.316968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.317238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.318032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:10.318337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.319829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.338106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 563/1094] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:10.343969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.370357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:10.379743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:10.415076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b373fac-ad96-49ac-978e-eea73795c029 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.443105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580a142-5d2a-4845-be19-041914cb594f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.451646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.452295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.452586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.452852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.463410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:10.468377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:10.469396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:10.470280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 532/1095] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:10.475520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.476458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.477075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.477362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.477634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.543691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.545162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.572400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:10.574739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753889-4d6f-4537-8d45-43f54efb374b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.601408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:10.610145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.613434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.613434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.613434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.613434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:10.613434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.613796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.622240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 533/1096] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:10.627772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.649362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 564/1097] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:43:10.654895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.730896 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e2c3-d2c0-40c0-b229-7539842ac6d9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.753257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646f0c0-b5c3-4344-88cc-0013f8adbd16 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.765428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.766072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.766356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.766636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] PATH_INFO: `/v3/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.767839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] RBAC: Authorizing `identity:get_user(user_id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.769283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.770892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 534/1098] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/users/84c54261d4fb4d8fb2cfeaa8fa7cf4e3 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:10.776593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.779389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:10.787899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.789067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.789353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.790686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.799832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.801291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:10.816861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:10.817959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:10.819240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:43:10.819240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:10.819240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:43:10.819240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:10.819240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 565/1099] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:10.824035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.837012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:10.880706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fb83e-7992-48a0-a22e-fa1fac8cccce None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.914581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.915204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.915483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.915806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.924231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c22c18-8eab-4e83-a028-8d0ac4ff09ae None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:10.956861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:10.957470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:10.957755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:10.958030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:10.958711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:10.959015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.961447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:10.973066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 566/1100] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:10.979433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:10.983203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:10.984706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:11.018234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:11.038933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:11.078961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e530c0-bb1c-4461-b88e-26a90d308570 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:11.082952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 535/1101] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:43:11.089691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:11.114983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:11.115697 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:11.115984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:11.116420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:11.122702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:11.124185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:43:11.136494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:11.137434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:11.138394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 567/1102] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:43:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:43:11.143466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:11.187175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb678b-acc3-43ca-bd1c-ffa55be321a9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:11.221070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:11.221813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:11.222101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:11.222380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:11.223613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] RBAC: Authorizing `identity:get_project(project_id=18945f05f1b24a21b755146f1f3b565c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:11.225056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:11.227610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 536/1103] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:11 2022] GET /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:11.233404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:11.240853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:11.244359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112aac1-2a93-459a-9695-d742b5a91ef2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:11.258934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:11.278568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:11.279185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:11.279491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:11.279791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:11.280549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:11.280854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:11.282290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:11.294407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 568/1104] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:11.300588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:11.333064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c947d-e5fd-4ba3-aa20-2f4ad83b9617 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:11.367344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f0862-9757-4f63-a7b1-aa33dd984fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:11.367983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0862-9757-4f63-a7b1-aa33dd984fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:11.368263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0862-9757-4f63-a7b1-aa33dd984fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:11.368534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0862-9757-4f63-a7b1-aa33dd984fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:11.369313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f0862-9757-4f63-a7b1-aa33dd984fa5 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=18945f05f1b24a21b755146f1f3b565c, user.id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:11.369621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f0862-9757-4f63-a7b1-aa33dd984fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:11.371039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f0862-9757-4f63-a7b1-aa33dd984fa5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:11.376688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 537/1105] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:11 2022] GET /identity/v3/role_assignments?user.id=84c54261d4fb4d8fb2cfeaa8fa7cf4e3&scope.project.id=18945f05f1b24a21b755146f1f3b565c&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:11.408435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13feac-52b2-41f6-afbf-9a6d53f43805 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:11.444684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafad39-b041-4978-8044-d81961307f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:11.445601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafad39-b041-4978-8044-d81961307f59 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:11.446012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafad39-b041-4978-8044-d81961307f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:11.446292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafad39-b041-4978-8044-d81961307f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:11.447114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafad39-b041-4978-8044-d81961307f59 None admin] RBAC: Adding query filter params (role.id=1d2b3f7e9adb449db4663a3dba84a1a4, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=659b367f1ce643a68d55ff1c00d8b7b8) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:11.447429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafad39-b041-4978-8044-d81961307f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:11.448923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafad39-b041-4978-8044-d81961307f59 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:11.455487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 569/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:11 2022] GET /identity/v3/role_assignments?user.id=659b367f1ce643a68d55ff1c00d8b7b8&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=1d2b3f7e9adb449db4663a3dba84a1a4 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:13.424178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716779d1-6474-4c35-bf4b-3aed327e4da6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:13.424732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716779d1-6474-4c35-bf4b-3aed327e4da6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:13.424732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716779d1-6474-4c35-bf4b-3aed327e4da6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:13.426072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 538/1107] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:13.431792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a8751-bab8-430d-830c-0843cdf2fb39 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:13.432164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a8751-bab8-430d-830c-0843cdf2fb39 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:13.432353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a8751-bab8-430d-830c-0843cdf2fb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:13.450966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cf5a8751-bab8-430d-830c-0843cdf2fb39 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:13.452193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cf5a8751-bab8-430d-830c-0843cdf2fb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:13.468468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:13.517500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 570/1108] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:13.541489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716779d1-6474-4c35-bf4b-3aed327e4da6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:13.631480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:13.645439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716779d1-6474-4c35-bf4b-3aed327e4da6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:13.678340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215d8d-8500-4c5f-a53f-096f895ba576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:13.679582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215d8d-8500-4c5f-a53f-096f895ba576 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:13.679983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215d8d-8500-4c5f-a53f-096f895ba576 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:13.680208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215d8d-8500-4c5f-a53f-096f895ba576 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:13.681364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57215d8d-8500-4c5f-a53f-096f895ba576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:13.682813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57215d8d-8500-4c5f-a53f-096f895ba576 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:13.684431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 539/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:13.690213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a8751-bab8-430d-830c-0843cdf2fb39 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:13.788073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a8751-bab8-430d-830c-0843cdf2fb39 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:13.821212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a52f6-4c54-4344-b1dd-b391aace8089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:13.821826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a52f6-4c54-4344-b1dd-b391aace8089 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:13.822113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a52f6-4c54-4344-b1dd-b391aace8089 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:13.822391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a52f6-4c54-4344-b1dd-b391aace8089 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:13.823071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a52f6-4c54-4344-b1dd-b391aace8089 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:13.824535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a52f6-4c54-4344-b1dd-b391aace8089 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:13.909834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 571/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:43:13 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:15.936879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad96f-f8b9-43b1-a4c7-495ba8ccf371 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:15.937503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad96f-f8b9-43b1-a4c7-495ba8ccf371 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:15.937503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad96f-f8b9-43b1-a4c7-495ba8ccf371 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:15.938787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 540/1111] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:15.943862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1858c-f90b-4071-8f61-7175a13fcdee None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:15.944320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1858c-f90b-4071-8f61-7175a13fcdee None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:15.944438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1858c-f90b-4071-8f61-7175a13fcdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:15.962687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f7b1858c-f90b-4071-8f61-7175a13fcdee None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:15.963902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f7b1858c-f90b-4071-8f61-7175a13fcdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:15.965791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:16.029860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 572/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:16.052295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad96f-f8b9-43b1-a4c7-495ba8ccf371 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:16.152693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad96f-f8b9-43b1-a4c7-495ba8ccf371 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:16.186853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:16.187478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:16.187790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:16.188068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] PATH_INFO: `/v3/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:16.189561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] RBAC: Authorizing `identity:get_role(role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:16.190941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:16.192303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 541/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:16 2022] GET /identity/v3/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:16.198469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1858c-f90b-4071-8f61-7175a13fcdee None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:16.295601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1858c-f90b-4071-8f61-7175a13fcdee None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:16.331285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:16.332077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:16.332511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:16.332632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] PATH_INFO: `/v3/users/32df852e49f349f29248740def2e2d1f` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:16.342312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:16.348545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] RBAC: Authorizing `identity:get_user(user_id=32df852e49f349f29248740def2e2d1f)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:16.350063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:16.357312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 573/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:16 2022] GET /identity/v3/users/32df852e49f349f29248740def2e2d1f => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:16.358351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:16.384452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:16.457631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0c68a-ac8d-40cd-bb95-6ec0f0e10464 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:16.490022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:16.490627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:16.490955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:16.491822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:16.556451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:16.557838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:16.580929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:16.654234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 542/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:16.660234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:16.758422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395cc16-9afb-49bb-aa67-a7381efde135 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:16.791856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed987fde-910b-4681-807d-e409b67f996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:16.792482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed987fde-910b-4681-807d-e409b67f996d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:16.792761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed987fde-910b-4681-807d-e409b67f996d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:16.793033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed987fde-910b-4681-807d-e409b67f996d None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:16.794173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed987fde-910b-4681-807d-e409b67f996d None admin] RBAC: Authorizing `identity:get_project(project_id=18945f05f1b24a21b755146f1f3b565c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:16.795623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed987fde-910b-4681-807d-e409b67f996d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:16.797252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 574/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:16 2022] GET /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:16.803368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:16.900948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec3254-fac2-43c3-a6bd-06d9fb061094 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:16.923125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:16.933377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9cc18-90eb-4b1e-b7fe-24e7ec031341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:16.933986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9cc18-90eb-4b1e-b7fe-24e7ec031341 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:16.934268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9cc18-90eb-4b1e-b7fe-24e7ec031341 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:16.934539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9cc18-90eb-4b1e-b7fe-24e7ec031341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:16.935319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9cc18-90eb-4b1e-b7fe-24e7ec031341 None admin] RBAC: Adding query filter params (role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=18945f05f1b24a21b755146f1f3b565c, user.id=32df852e49f349f29248740def2e2d1f) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:16.935660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9cc18-90eb-4b1e-b7fe-24e7ec031341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:16.937072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9cc18-90eb-4b1e-b7fe-24e7ec031341 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:16.942238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 543/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:16 2022] GET /identity/v3/role_assignments?user.id=32df852e49f349f29248740def2e2d1f&scope.project.id=18945f05f1b24a21b755146f1f3b565c&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:18.899897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53d743-276e-434a-b92c-3ddd101baf31 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:18.900408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53d743-276e-434a-b92c-3ddd101baf31 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:18.900501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53d743-276e-434a-b92c-3ddd101baf31 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:18.901797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 575/1118] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:18.906799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315a6e-1e74-4c61-8660-c6ce86a5be8f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:18.907183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315a6e-1e74-4c61-8660-c6ce86a5be8f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:18.907353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315a6e-1e74-4c61-8660-c6ce86a5be8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:18.925669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8e315a6e-1e74-4c61-8660-c6ce86a5be8f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:18.926846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8e315a6e-1e74-4c61-8660-c6ce86a5be8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:18.992379 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 544/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:19.014743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53d743-276e-434a-b92c-3ddd101baf31 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:19.076781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:19.114498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53d743-276e-434a-b92c-3ddd101baf31 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:19.148775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:19.149400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:19.149683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:19.149957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] PATH_INFO: `/v3/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:19.151383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] RBAC: Authorizing `identity:get_role(role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:19.152784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:19.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 576/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:19 2022] GET /identity/v3/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:19.159918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315a6e-1e74-4c61-8660-c6ce86a5be8f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:19.256982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315a6e-1e74-4c61-8660-c6ce86a5be8f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:19.273723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:19.290349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:19.290982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:19.291291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:19.291589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] PATH_INFO: `/v3/users/32df852e49f349f29248740def2e2d1f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:19.292747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] RBAC: Authorizing `identity:get_user(user_id=32df852e49f349f29248740def2e2d1f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:19.294172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:19.295826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 545/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:19 2022] GET /identity/v3/users/32df852e49f349f29248740def2e2d1f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:19.301519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:19.401072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa646a8c-efe1-42ea-b26e-6d95a8e475da None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:19.429776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:19.434705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155408c1-3c78-4ddc-8232-ca42a1cd3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:19.435345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155408c1-3c78-4ddc-8232-ca42a1cd3efd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:19.435663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155408c1-3c78-4ddc-8232-ca42a1cd3efd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:19.435945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155408c1-3c78-4ddc-8232-ca42a1cd3efd None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:19.437114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155408c1-3c78-4ddc-8232-ca42a1cd3efd None admin] RBAC: Authorizing `identity:get_project(project_id=18945f05f1b24a21b755146f1f3b565c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:19.438560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155408c1-3c78-4ddc-8232-ca42a1cd3efd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:19.443668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 577/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:19 2022] GET /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:19.446547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:19.543964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a50d4-dcb6-41df-857e-a7753c485b16 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:19.576036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc2c39-68f1-435e-9533-ca261fef5861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:19.576689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc2c39-68f1-435e-9533-ca261fef5861 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:19.576967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc2c39-68f1-435e-9533-ca261fef5861 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:19.577241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc2c39-68f1-435e-9533-ca261fef5861 None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c/users/32df852e49f349f29248740def2e2d1f/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:19.579361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc2c39-68f1-435e-9533-ca261fef5861 None admin] RBAC: Authorizing `identity:create_grant(project_id=18945f05f1b24a21b755146f1f3b565c, user_id=32df852e49f349f29248740def2e2d1f, role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:19.580813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc2c39-68f1-435e-9533-ca261fef5861 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:19.646195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 546/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:43:19 2022] PUT /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c/users/32df852e49f349f29248740def2e2d1f/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:21.642161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac79761-18dd-4f08-a994-dd210cd5a306 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:21.642797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac79761-18dd-4f08-a994-dd210cd5a306 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:21.642797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac79761-18dd-4f08-a994-dd210cd5a306 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:21.644191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 578/1124] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:21.650128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ba219-cb86-452a-9050-33ea3fb9cace None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:21.650532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ba219-cb86-452a-9050-33ea3fb9cace None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:21.650681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ba219-cb86-452a-9050-33ea3fb9cace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:21.669057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f55ba219-cb86-452a-9050-33ea3fb9cace None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:21.670241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f55ba219-cb86-452a-9050-33ea3fb9cace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:21.716044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:21.739109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 547/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:21.764030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac79761-18dd-4f08-a994-dd210cd5a306 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:21.866337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac79761-18dd-4f08-a994-dd210cd5a306 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:21.898836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:21.899403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:21.899712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:21.899986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] PATH_INFO: `/v3/roles/41ad02f8e4774c4595f5ececec5fd55b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:21.901406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] RBAC: Authorizing `identity:get_role(role_id=41ad02f8e4774c4595f5ececec5fd55b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:21.902769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:21.904109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 579/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:21 2022] GET /identity/v3/roles/41ad02f8e4774c4595f5ececec5fd55b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:21.909967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ba219-cb86-452a-9050-33ea3fb9cace None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:22.008207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ba219-cb86-452a-9050-33ea3fb9cace None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:22.041458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:22.042089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:22.042375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:22.042652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] PATH_INFO: `/v3/users/32df852e49f349f29248740def2e2d1f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:22.043833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] RBAC: Authorizing `identity:get_user(user_id=32df852e49f349f29248740def2e2d1f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:22.045355 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:22.046949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 548/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:21 2022] GET /identity/v3/users/32df852e49f349f29248740def2e2d1f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:22.052685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:22.118015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:22.149489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d8cef-bdac-42b8-992f-2532715076bb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:22.182970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:22.183611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:22.183895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:22.184167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:22.248305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:22.249735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:22.311320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:22.345781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 580/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:22.351750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:22.396675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:22.449056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4bfd-fc69-48de-942f-1398199b6b27 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:22.481410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a10ab-fa80-4017-84a1-67742eb6f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:22.482042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a10ab-fa80-4017-84a1-67742eb6f7fa None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:22.482325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a10ab-fa80-4017-84a1-67742eb6f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:22.482595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a10ab-fa80-4017-84a1-67742eb6f7fa None admin] PATH_INFO: `/v3/projects/18945f05f1b24a21b755146f1f3b565c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:22.483765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a10ab-fa80-4017-84a1-67742eb6f7fa None admin] RBAC: Authorizing `identity:get_project(project_id=18945f05f1b24a21b755146f1f3b565c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:22.485193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a10ab-fa80-4017-84a1-67742eb6f7fa None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:22.486835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 549/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:22 2022] GET /identity/v3/projects/18945f05f1b24a21b755146f1f3b565c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:22.493654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:22.591177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a41ac-8583-4830-b0bf-2b372266f83b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:22.625035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ba55b-b893-4dca-abcc-d6183beb4c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:22.625653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba55b-b893-4dca-abcc-d6183beb4c4c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:22.625940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba55b-b893-4dca-abcc-d6183beb4c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:22.626221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba55b-b893-4dca-abcc-d6183beb4c4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:22.627004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba55b-b893-4dca-abcc-d6183beb4c4c None admin] RBAC: Adding query filter params (role.id=41ad02f8e4774c4595f5ececec5fd55b, scope.project.id=18945f05f1b24a21b755146f1f3b565c, user.id=32df852e49f349f29248740def2e2d1f) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:22.627319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba55b-b893-4dca-abcc-d6183beb4c4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:22.628783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba55b-b893-4dca-abcc-d6183beb4c4c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:22.635248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 581/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:22 2022] GET /identity/v3/role_assignments?user.id=32df852e49f349f29248740def2e2d1f&scope.project.id=18945f05f1b24a21b755146f1f3b565c&role.id=41ad02f8e4774c4595f5ececec5fd55b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:24.642749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5479e-5171-46f7-9587-1a888bef8aec None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:24.643311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5479e-5171-46f7-9587-1a888bef8aec None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:24.643311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5479e-5171-46f7-9587-1a888bef8aec None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:24.644673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 550/1131] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:24.650355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f14355-5d6c-4e1c-886d-709e1ea6df24 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:24.650745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f14355-5d6c-4e1c-886d-709e1ea6df24 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:24.650918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f14355-5d6c-4e1c-886d-709e1ea6df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:24.669329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-49f14355-5d6c-4e1c-886d-709e1ea6df24 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:24.670519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-49f14355-5d6c-4e1c-886d-709e1ea6df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:24.695413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:24.735730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 582/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:24.758690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5479e-5171-46f7-9587-1a888bef8aec None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:24.857783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5479e-5171-46f7-9587-1a888bef8aec None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:24.870591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:24.890686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac7d50-950c-4fa4-a0c2-4d1e000dfcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:24.891314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac7d50-950c-4fa4-a0c2-4d1e000dfcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:24.891637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac7d50-950c-4fa4-a0c2-4d1e000dfcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:24.891912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac7d50-950c-4fa4-a0c2-4d1e000dfcb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:24.893094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac7d50-950c-4fa4-a0c2-4d1e000dfcb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:24.894468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac7d50-950c-4fa4-a0c2-4d1e000dfcb3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:24.896077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 551/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:24.902586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f14355-5d6c-4e1c-886d-709e1ea6df24 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:24.999544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f14355-5d6c-4e1c-886d-709e1ea6df24 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:25.033109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e2d58-180e-4a88-afaf-b1f539484a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:25.033720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2d58-180e-4a88-afaf-b1f539484a12 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:25.034007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2d58-180e-4a88-afaf-b1f539484a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:25.034288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2d58-180e-4a88-afaf-b1f539484a12 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:25.034967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e2d58-180e-4a88-afaf-b1f539484a12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:25.036439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e2d58-180e-4a88-afaf-b1f539484a12 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:25.133805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 583/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:43:24 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:27.159829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806927b-21bd-4035-aeed-e85bdccdadfb None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.160348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806927b-21bd-4035-aeed-e85bdccdadfb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.160473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806927b-21bd-4035-aeed-e85bdccdadfb None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.161734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 552/1135] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:27.167518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73299a-f545-49d2-998b-232192af4d4d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.167983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73299a-f545-49d2-998b-232192af4d4d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.168085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73299a-f545-49d2-998b-232192af4d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.186188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-dc73299a-f545-49d2-998b-232192af4d4d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:27.187384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dc73299a-f545-49d2-998b-232192af4d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:27.196565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:27.253292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 584/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:27.278151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806927b-21bd-4035-aeed-e85bdccdadfb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.378391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806927b-21bd-4035-aeed-e85bdccdadfb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.412089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be980d0-8de4-4b7e-bf99-cd4f4b11bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.412662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be980d0-8de4-4b7e-bf99-cd4f4b11bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.412949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be980d0-8de4-4b7e-bf99-cd4f4b11bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.413226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be980d0-8de4-4b7e-bf99-cd4f4b11bfd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.414368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be980d0-8de4-4b7e-bf99-cd4f4b11bfd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.415841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be980d0-8de4-4b7e-bf99-cd4f4b11bfd7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.417425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 553/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:43:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:27.423199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73299a-f545-49d2-998b-232192af4d4d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.520363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73299a-f545-49d2-998b-232192af4d4d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.553357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2272da-6981-452d-b159-9397557cb797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.553966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2272da-6981-452d-b159-9397557cb797 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.554255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2272da-6981-452d-b159-9397557cb797 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.554533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2272da-6981-452d-b159-9397557cb797 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.555211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2272da-6981-452d-b159-9397557cb797 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.556696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2272da-6981-452d-b159-9397557cb797 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.630168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:27.635612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 585/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:43:27 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:29.665228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0786c-ab05-405a-ace1-beaa47898fa8 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:29.665794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0786c-ab05-405a-ace1-beaa47898fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:29.665794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0786c-ab05-405a-ace1-beaa47898fa8 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:29.667157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 554/1139] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:29.672313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15e33d-2711-4da3-a66c-24b3eddb2a6c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:29.672731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15e33d-2711-4da3-a66c-24b3eddb2a6c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:29.672865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15e33d-2711-4da3-a66c-24b3eddb2a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:29.691234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3e15e33d-2711-4da3-a66c-24b3eddb2a6c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:29.692463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3e15e33d-2711-4da3-a66c-24b3eddb2a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:29.756475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 586/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:29.779154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0786c-ab05-405a-ace1-beaa47898fa8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:29.830535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:29.881168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0786c-ab05-405a-ace1-beaa47898fa8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:29.914503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:29.915124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:29.915452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:29.915706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:29.925496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:29.930419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:29.931662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:29.932776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 555/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:29.937569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15e33d-2711-4da3-a66c-24b3eddb2a6c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:30.033438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15e33d-2711-4da3-a66c-24b3eddb2a6c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:30.061537 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:30.067224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:30.067868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:30.068161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:30.068430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:30.069146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:30.069453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:30.070825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:30.078895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 587/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:30.085355 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:30.182759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d873db0-b370-4452-8ed0-708280296a84 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:30.191665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:30.215901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:30.216720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:30.217214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:30.217543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] PATH_INFO: `/v3/users/774a19862f7e410c92b63d8fa4dd187a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:30.231732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] RBAC: Authorizing `identity:get_user(user_id=774a19862f7e410c92b63d8fa4dd187a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:30.233200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:30.234854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 556/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:30 2022] GET /identity/v3/users/774a19862f7e410c92b63d8fa4dd187a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:30.240738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:30.341247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e7cd3-bd43-48f0-a0c3-d87a22d17024 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:30.373865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:30.374480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:30.374768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:30.375047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:30.431875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:30.440968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:30.442429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:30.538891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 588/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:30.544734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:30.642860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a8055-0a91-45f1-a712-81fb86a978c4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:30.675618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97c661-e43b-4e74-b48f-fcf85569bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:30.676250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97c661-e43b-4e74-b48f-fcf85569bf2d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:30.676536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97c661-e43b-4e74-b48f-fcf85569bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:30.676812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97c661-e43b-4e74-b48f-fcf85569bf2d None admin] PATH_INFO: `/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:30.678454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97c661-e43b-4e74-b48f-fcf85569bf2d None admin] RBAC: Authorizing `identity:get_project(project_id=0cc8e553bbf34ba0b555631f5e2ff7f9)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:30.679912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97c661-e43b-4e74-b48f-fcf85569bf2d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:30.681559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 557/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:30 2022] GET /identity/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:30.687539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:30.776452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:30.784826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff7b58-acaf-48b1-92b5-13fbc2812cc9 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:30.819149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4047f-863d-41e2-b8b1-7e53f84ee0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:30.819782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4047f-863d-41e2-b8b1-7e53f84ee0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:30.820067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4047f-863d-41e2-b8b1-7e53f84ee0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:30.820342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4047f-863d-41e2-b8b1-7e53f84ee0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:30.821121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4047f-863d-41e2-b8b1-7e53f84ee0b0 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=0cc8e553bbf34ba0b555631f5e2ff7f9, user.id=774a19862f7e410c92b63d8fa4dd187a) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:30.821432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4047f-863d-41e2-b8b1-7e53f84ee0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:30.822860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4047f-863d-41e2-b8b1-7e53f84ee0b0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:30.828098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 589/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:30 2022] GET /identity/v3/role_assignments?user.id=774a19862f7e410c92b63d8fa4dd187a&scope.project.id=0cc8e553bbf34ba0b555631f5e2ff7f9&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:32.773302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f0b5-84d6-4570-b119-2ebb554aa430 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.773719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f0b5-84d6-4570-b119-2ebb554aa430 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.773856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f0b5-84d6-4570-b119-2ebb554aa430 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.775205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 558/1147] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:32.780374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70995e2d-e3fb-477d-a8f6-f276d94e4d86 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.780780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70995e2d-e3fb-477d-a8f6-f276d94e4d86 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.780933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70995e2d-e3fb-477d-a8f6-f276d94e4d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.799000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-70995e2d-e3fb-477d-a8f6-f276d94e4d86 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:32.800217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-70995e2d-e3fb-477d-a8f6-f276d94e4d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:32.865394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 590/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:32.887124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f0b5-84d6-4570-b119-2ebb554aa430 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.921983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:32.985235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f0b5-84d6-4570-b119-2ebb554aa430 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.017186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.017750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.018028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.018299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.028719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:33.033595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:33.034682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:33.035664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 559/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:33.041250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70995e2d-e3fb-477d-a8f6-f276d94e4d86 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.122814 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:33.137299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70995e2d-e3fb-477d-a8f6-f276d94e4d86 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.171108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.171748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.172034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.172311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.173017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:33.175477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.175477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.183170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 591/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:33.188872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.280496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:33.287490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5642157-bf70-47f1-a2c8-0bfcfde0c9bc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.321486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.322111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.322389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.322659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] PATH_INFO: `/v3/users/774a19862f7e410c92b63d8fa4dd187a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.323827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] RBAC: Authorizing `identity:get_user(user_id=774a19862f7e410c92b63d8fa4dd187a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.325244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.326805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 560/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:33 2022] GET /identity/v3/users/774a19862f7e410c92b63d8fa4dd187a => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:33.332618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.433431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20beb8-8bf8-48d8-a200-82f30fb18f4a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.465893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65e888-74b9-4d4c-8dfc-28ec63b2c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.466526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65e888-74b9-4d4c-8dfc-28ec63b2c461 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.466816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65e888-74b9-4d4c-8dfc-28ec63b2c461 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.467097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65e888-74b9-4d4c-8dfc-28ec63b2c461 None admin] PATH_INFO: `/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.468272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65e888-74b9-4d4c-8dfc-28ec63b2c461 None admin] RBAC: Authorizing `identity:get_project(project_id=0cc8e553bbf34ba0b555631f5e2ff7f9)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.469716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65e888-74b9-4d4c-8dfc-28ec63b2c461 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.471367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 592/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:33 2022] GET /identity/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:33.477706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.573823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e09f61-3b6a-4e39-a085-dc8573b7ade2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.606978 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2a8d6-39d0-44e1-b520-afa9edcff49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.607651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2a8d6-39d0-44e1-b520-afa9edcff49a None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.607933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2a8d6-39d0-44e1-b520-afa9edcff49a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.608206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2a8d6-39d0-44e1-b520-afa9edcff49a None admin] PATH_INFO: `/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9/users/774a19862f7e410c92b63d8fa4dd187a/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.610103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2a8d6-39d0-44e1-b520-afa9edcff49a None admin] RBAC: Authorizing `identity:create_grant(project_id=0cc8e553bbf34ba0b555631f5e2ff7f9, user_id=774a19862f7e410c92b63d8fa4dd187a, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.611515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2a8d6-39d0-44e1-b520-afa9edcff49a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.720479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 561/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:43:33 2022] PUT /identity/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9/users/774a19862f7e410c92b63d8fa4dd187a/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:35.696366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc127a7-1452-4417-8425-7bf372ea3e6d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:35.696929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc127a7-1452-4417-8425-7bf372ea3e6d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:35.696929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc127a7-1452-4417-8425-7bf372ea3e6d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:35.698279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 593/1154] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:35.703340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41669e40-a19b-4c53-8cf9-65d9cdf21254 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:35.703748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41669e40-a19b-4c53-8cf9-65d9cdf21254 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:35.703927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41669e40-a19b-4c53-8cf9-65d9cdf21254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:35.722881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-41669e40-a19b-4c53-8cf9-65d9cdf21254 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:35.724158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-41669e40-a19b-4c53-8cf9-65d9cdf21254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:35.740300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:35.791005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 562/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:35.814774 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc127a7-1452-4417-8425-7bf372ea3e6d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:35.832858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:35.915361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc127a7-1452-4417-8425-7bf372ea3e6d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:35.947812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:35.948379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:35.948659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:35.948929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:35.959420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:35.964298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:35.965434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:35.966471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 594/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:35.971612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41669e40-a19b-4c53-8cf9-65d9cdf21254 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:36.068676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41669e40-a19b-4c53-8cf9-65d9cdf21254 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:36.102532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:36.103143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:36.103428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:36.103737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:36.104450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:36.104759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:36.106126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:36.114055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 563/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:36.119889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:36.195459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:36.218940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c0940-c53d-4afa-bc27-016cabc0f3f5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:36.252430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:36.253065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:36.253350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:36.253626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] PATH_INFO: `/v3/users/774a19862f7e410c92b63d8fa4dd187a` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:36.254780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] RBAC: Authorizing `identity:get_user(user_id=774a19862f7e410c92b63d8fa4dd187a)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:36.256746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:36.258331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 595/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:36 2022] GET /identity/v3/users/774a19862f7e410c92b63d8fa4dd187a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:36.264194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:36.280871 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:36.363666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96bc7a-d483-4291-a17f-773ba2fa0a4a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:36.398031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:36.398641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:36.398953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:36.399231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:36.465192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:36.466629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:36.484321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:36.562444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 564/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:36.569050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:36.666870 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc779576-46b0-4523-b845-28ba65dddd86 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:36.700221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1664f12-b88e-45f5-8f34-88cdcff1adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:36.700851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1664f12-b88e-45f5-8f34-88cdcff1adbf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:36.701133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1664f12-b88e-45f5-8f34-88cdcff1adbf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:36.701408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1664f12-b88e-45f5-8f34-88cdcff1adbf None admin] PATH_INFO: `/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:36.702561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1664f12-b88e-45f5-8f34-88cdcff1adbf None admin] RBAC: Authorizing `identity:get_project(project_id=0cc8e553bbf34ba0b555631f5e2ff7f9)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:36.704038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1664f12-b88e-45f5-8f34-88cdcff1adbf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:36.705682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 596/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:36 2022] GET /identity/v3/projects/0cc8e553bbf34ba0b555631f5e2ff7f9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:36.712304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:36.826314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a8a61-2e36-4001-af6e-7175fe5341d7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:36.840119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:36.859552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f905fa-0b0a-4272-aad7-c4f98dd6012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:36.860170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f905fa-0b0a-4272-aad7-c4f98dd6012c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:36.860454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f905fa-0b0a-4272-aad7-c4f98dd6012c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:36.860727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f905fa-0b0a-4272-aad7-c4f98dd6012c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:36.861553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f905fa-0b0a-4272-aad7-c4f98dd6012c None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=0cc8e553bbf34ba0b555631f5e2ff7f9, user.id=774a19862f7e410c92b63d8fa4dd187a) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:36.861863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f905fa-0b0a-4272-aad7-c4f98dd6012c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:36.863280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f905fa-0b0a-4272-aad7-c4f98dd6012c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:36.869406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 565/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:36 2022] GET /identity/v3/role_assignments?user.id=774a19862f7e410c92b63d8fa4dd187a&scope.project.id=0cc8e553bbf34ba0b555631f5e2ff7f9&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:38.863549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018687fb-89fc-41fb-9ced-6b8b713290dc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:38.864042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018687fb-89fc-41fb-9ced-6b8b713290dc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:38.864173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018687fb-89fc-41fb-9ced-6b8b713290dc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:38.866066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 597/1162] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:38.871249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a3890-056e-4c83-8b05-f66ab8bc8cf6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:38.871693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a3890-056e-4c83-8b05-f66ab8bc8cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:38.871853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a3890-056e-4c83-8b05-f66ab8bc8cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:38.890470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d00a3890-056e-4c83-8b05-f66ab8bc8cf6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:38.891697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d00a3890-056e-4c83-8b05-f66ab8bc8cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:38.956696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 566/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:38.978797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018687fb-89fc-41fb-9ced-6b8b713290dc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:38.994458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:39.079599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018687fb-89fc-41fb-9ced-6b8b713290dc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:39.112361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:39.112996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:39.113281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:39.113564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:39.179000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:39.180495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:39.194051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:39.276828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 598/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:39.282785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a3890-056e-4c83-8b05-f66ab8bc8cf6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:39.380546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a3890-056e-4c83-8b05-f66ab8bc8cf6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:39.389427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:39.414005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420be3c-4139-41ac-9542-4edf58329c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:39.414636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420be3c-4139-41ac-9542-4edf58329c7b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:39.414921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420be3c-4139-41ac-9542-4edf58329c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:39.415198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420be3c-4139-41ac-9542-4edf58329c7b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:39.421793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420be3c-4139-41ac-9542-4edf58329c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:39.423226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420be3c-4139-41ac-9542-4edf58329c7b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0420be3c-4139-41ac-9542-4edf58329c7b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:39.435270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:39.436373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:39.437247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 567/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:43:39 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:39.442528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:39.540227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1157c06-ba19-4ed4-90a6-9eecfc8f2177 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:39.553061 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:39.573681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6442-5a25-4d3f-80b8-11aa43789243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:39.574296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6442-5a25-4d3f-80b8-11aa43789243 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:39.574583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6442-5a25-4d3f-80b8-11aa43789243 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:39.574867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6442-5a25-4d3f-80b8-11aa43789243 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:39.575587 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6442-5a25-4d3f-80b8-11aa43789243 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:39.575900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6442-5a25-4d3f-80b8-11aa43789243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:39.577603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6442-5a25-4d3f-80b8-11aa43789243 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:39.585810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 599/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 07:43:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:41.585643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611eb4e-face-495b-9c35-8667b9518cc2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:41.586214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611eb4e-face-495b-9c35-8667b9518cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:41.586214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611eb4e-face-495b-9c35-8667b9518cc2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:41.587564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 568/1167] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:41.592769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9047ddb-b7d4-4f5b-87d7-a78e20de3552 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:41.593196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9047ddb-b7d4-4f5b-87d7-a78e20de3552 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:41.593330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9047ddb-b7d4-4f5b-87d7-a78e20de3552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:41.611707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d9047ddb-b7d4-4f5b-87d7-a78e20de3552 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:41.612881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d9047ddb-b7d4-4f5b-87d7-a78e20de3552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:41.677050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 600/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:41.700640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611eb4e-face-495b-9c35-8667b9518cc2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:41.801195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611eb4e-face-495b-9c35-8667b9518cc2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:41.834426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f067237-cd9b-4553-b79d-6f9195240271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:41.835029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f067237-cd9b-4553-b79d-6f9195240271 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:41.835307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f067237-cd9b-4553-b79d-6f9195240271 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:41.835604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f067237-cd9b-4553-b79d-6f9195240271 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:41.836356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f067237-cd9b-4553-b79d-6f9195240271 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:41.837658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f067237-cd9b-4553-b79d-6f9195240271 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:41.932098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 569/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Mon Aug 1 07:43:41 2022] POST /identity/v3/domains => generated 267 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:43.969256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daecb0c-5ea8-42f3-8ec7-ea920ce5ec12 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:43.969813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daecb0c-5ea8-42f3-8ec7-ea920ce5ec12 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:43.969813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daecb0c-5ea8-42f3-8ec7-ea920ce5ec12 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:43.971172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 601/1170] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:43.976464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f7320-e3a0-4dfb-9739-1d7596099644 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:43.976858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f7320-e3a0-4dfb-9739-1d7596099644 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:43.977026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f7320-e3a0-4dfb-9739-1d7596099644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:43.994842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c01f7320-e3a0-4dfb-9739-1d7596099644 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:43.996067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c01f7320-e3a0-4dfb-9739-1d7596099644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:44.018902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:44.061512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 570/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:44.085300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daecb0c-5ea8-42f3-8ec7-ea920ce5ec12 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:44.185807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daecb0c-5ea8-42f3-8ec7-ea920ce5ec12 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:44.194292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:44.219964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f48b1-7c09-434a-accc-ea76375a073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:44.220597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f48b1-7c09-434a-accc-ea76375a073b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:44.220881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f48b1-7c09-434a-accc-ea76375a073b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:44.221170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f48b1-7c09-434a-accc-ea76375a073b None admin] PATH_INFO: `/v3/domains/601983cef6a04717856f6c0641e044b7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:44.231763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f48b1-7c09-434a-accc-ea76375a073b None admin] RBAC: Authorizing `identity:get_domain(domain_id=601983cef6a04717856f6c0641e044b7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:44.233217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f48b1-7c09-434a-accc-ea76375a073b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:44.234830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 602/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 1 07:43:44 2022] GET /identity/v3/domains/601983cef6a04717856f6c0641e044b7 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:44.241391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f7320-e3a0-4dfb-9739-1d7596099644 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:44.337865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f7320-e3a0-4dfb-9739-1d7596099644 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:44.370217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe2e3b-4f18-45a6-a8be-3a0a69fc1e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:44.370814 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe2e3b-4f18-45a6-a8be-3a0a69fc1e57 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:44.371099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe2e3b-4f18-45a6-a8be-3a0a69fc1e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:44.371367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe2e3b-4f18-45a6-a8be-3a0a69fc1e57 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:44.372074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe2e3b-4f18-45a6-a8be-3a0a69fc1e57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:44.373495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe2e3b-4f18-45a6-a8be-3a0a69fc1e57 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:44.452170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 571/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:43:44 2022] POST /identity/v3/projects => generated 361 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:43:46.476439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6562e5-98a4-4afb-a983-53dc02ba7cb6 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.477009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6562e5-98a4-4afb-a983-53dc02ba7cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.477009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6562e5-98a4-4afb-a983-53dc02ba7cb6 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.478347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 603/1174] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:46.483556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59f6d-9389-4025-b3e0-b97f9d1f3f0b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.483965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59f6d-9389-4025-b3e0-b97f9d1f3f0b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.484101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59f6d-9389-4025-b3e0-b97f9d1f3f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.502395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1dd59f6d-9389-4025-b3e0-b97f9d1f3f0b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:46.503610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1dd59f6d-9389-4025-b3e0-b97f9d1f3f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:46.506332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:46.569310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 572/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:46.593411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6562e5-98a4-4afb-a983-53dc02ba7cb6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.693513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6562e5-98a4-4afb-a983-53dc02ba7cb6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.728696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f7849-7e5c-4418-bf58-98095ffb949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.729264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7849-7e5c-4418-bf58-98095ffb949e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.729550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7849-7e5c-4418-bf58-98095ffb949e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.729831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7849-7e5c-4418-bf58-98095ffb949e None admin] PATH_INFO: `/v3/domains/601983cef6a04717856f6c0641e044b7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.731028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f7849-7e5c-4418-bf58-98095ffb949e None admin] RBAC: Authorizing `identity:get_domain(domain_id=601983cef6a04717856f6c0641e044b7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.732512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f7849-7e5c-4418-bf58-98095ffb949e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.734098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 604/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 1 07:43:46 2022] GET /identity/v3/domains/601983cef6a04717856f6c0641e044b7 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:46.740448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd59f6d-9389-4025-b3e0-b97f9d1f3f0b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.837687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd59f6d-9389-4025-b3e0-b97f9d1f3f0b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.871161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eb1b6-3c29-4834-841e-310fd2adffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.871813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb1b6-3c29-4834-841e-310fd2adffa5 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.872099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb1b6-3c29-4834-841e-310fd2adffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.872376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb1b6-3c29-4834-841e-310fd2adffa5 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.873068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870eb1b6-3c29-4834-841e-310fd2adffa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.874495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870eb1b6-3c29-4834-841e-310fd2adffa5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.885265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:46.987590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 573/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/users => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:49.015199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fb6ce-47c5-48e8-93a5-877ed0d59741 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.015729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fb6ce-47c5-48e8-93a5-877ed0d59741 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.015729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fb6ce-47c5-48e8-93a5-877ed0d59741 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.017089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 605/1178] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:49.022789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2542-405d-4072-a2ab-68d7bc41da89 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.023233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2542-405d-4072-a2ab-68d7bc41da89 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.023335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2542-405d-4072-a2ab-68d7bc41da89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.042009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f86f2542-405d-4072-a2ab-68d7bc41da89 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:49.043179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f86f2542-405d-4072-a2ab-68d7bc41da89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:49.107624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 574/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:49.130241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fb6ce-47c5-48e8-93a5-877ed0d59741 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:49.165645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:49.230430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fb6ce-47c5-48e8-93a5-877ed0d59741 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:49.262334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:49.262964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.263249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.263524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.274366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:49.279200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:49.280327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:49.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 606/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:49.287110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2542-405d-4072-a2ab-68d7bc41da89 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:49.383745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2542-405d-4072-a2ab-68d7bc41da89 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:49.408746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:49.417183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:49.427468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.427468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.427468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.427468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:49.427468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:49.427468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:49.428532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 575/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:49.434202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:49.526158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:49.532574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8d220-0c38-41a6-a490-ef56c0350e3e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:49.565031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:49.565666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.565952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.566231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] PATH_INFO: `/v3/users/8e960b70ef1f420582557a38b71c2904` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.579402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] RBAC: Authorizing `identity:get_user(user_id=8e960b70ef1f420582557a38b71c2904)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:49.580889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:49.583302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 607/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:49 2022] GET /identity/v3/users/8e960b70ef1f420582557a38b71c2904 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:49.589121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:49.688250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24786c-2269-44eb-bd78-a534364b67da None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:49.721059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:49.721666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.721954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.722231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.779708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:49.792265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:49.793801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:49.887603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 576/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:49.893797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:49.991906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea17bf-c803-4482-8d5d-2fcfbf5ceaf7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:50.025321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00080705-93e8-475b-99cd-248b05983554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:50.025950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080705-93e8-475b-99cd-248b05983554 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:50.026230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080705-93e8-475b-99cd-248b05983554 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:50.026500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080705-93e8-475b-99cd-248b05983554 None admin] PATH_INFO: `/v3/projects/dfc069bb73144aedb17c6ed3f5373d90` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:50.027663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00080705-93e8-475b-99cd-248b05983554 None admin] RBAC: Authorizing `identity:get_project(project_id=dfc069bb73144aedb17c6ed3f5373d90)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:50.029099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00080705-93e8-475b-99cd-248b05983554 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:50.030777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 608/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:49 2022] GET /identity/v3/projects/dfc069bb73144aedb17c6ed3f5373d90 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:50.037580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:50.121500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:50.133602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23a67-ff1d-4b23-aab7-68502848faf3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:50.166957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917855e-94db-459d-8285-00246be73d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:50.167594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917855e-94db-459d-8285-00246be73d8f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:50.167879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917855e-94db-459d-8285-00246be73d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:50.168152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917855e-94db-459d-8285-00246be73d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:50.168930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917855e-94db-459d-8285-00246be73d8f None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=dfc069bb73144aedb17c6ed3f5373d90, user.id=8e960b70ef1f420582557a38b71c2904) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:50.169237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917855e-94db-459d-8285-00246be73d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:50.170659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917855e-94db-459d-8285-00246be73d8f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:50.175936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 577/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:50 2022] GET /identity/v3/role_assignments?user.id=8e960b70ef1f420582557a38b71c2904&scope.project.id=dfc069bb73144aedb17c6ed3f5373d90&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:52.151719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78c96-edb3-471f-b1cc-503107b6cdfe None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.152346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78c96-edb3-471f-b1cc-503107b6cdfe None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.152346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78c96-edb3-471f-b1cc-503107b6cdfe None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.153631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 609/1186] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:52.158769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f89a-7b56-49ff-ac42-caa3904dd0f7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.159199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f89a-7b56-49ff-ac42-caa3904dd0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.159322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f89a-7b56-49ff-ac42-caa3904dd0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.177978 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2f58f89a-7b56-49ff-ac42-caa3904dd0f7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:52.179168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2f58f89a-7b56-49ff-ac42-caa3904dd0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:52.244801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 578/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:52.267310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be78c96-edb3-471f-b1cc-503107b6cdfe None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:52.288480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:52.370205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be78c96-edb3-471f-b1cc-503107b6cdfe None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:52.403884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:52.404457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.404742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.405017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.415497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:52.420450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:52.421603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:43:52.422674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 610/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:52.428781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58f89a-7b56-49ff-ac42-caa3904dd0f7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:52.509486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:52.527127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58f89a-7b56-49ff-ac42-caa3904dd0f7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:52.561942 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:52.562567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.562853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.563130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.563881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:52.564191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:52.565571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:52.573674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 579/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:52.579330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:52.658738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:52.678312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc806-7de2-4ac7-9721-da1cb4fc7413 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:52.711496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:52.712163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.712448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.712724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] PATH_INFO: `/v3/users/8e960b70ef1f420582557a38b71c2904` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.713920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] RBAC: Authorizing `identity:get_user(user_id=8e960b70ef1f420582557a38b71c2904)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:52.715359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:52.716991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 611/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:52 2022] GET /identity/v3/users/8e960b70ef1f420582557a38b71c2904 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:52.723208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:52.820459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11216ffa-28c9-47ba-be46-38995e40605b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:52.853799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1136a-47a7-4fe3-847c-ceb9f906d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:52.854430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1136a-47a7-4fe3-847c-ceb9f906d89a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.854713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1136a-47a7-4fe3-847c-ceb9f906d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.854987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1136a-47a7-4fe3-847c-ceb9f906d89a None admin] PATH_INFO: `/v3/projects/dfc069bb73144aedb17c6ed3f5373d90` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.856154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1136a-47a7-4fe3-847c-ceb9f906d89a None admin] RBAC: Authorizing `identity:get_project(project_id=dfc069bb73144aedb17c6ed3f5373d90)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:52.857583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1136a-47a7-4fe3-847c-ceb9f906d89a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:52.859228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 580/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:52 2022] GET /identity/v3/projects/dfc069bb73144aedb17c6ed3f5373d90 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:52.865250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:52.963568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66c249-fae4-42fc-8a6e-08a4355823b3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:52.996566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b196f48-f917-4882-91d3-4a96b90e4d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:52.997221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196f48-f917-4882-91d3-4a96b90e4d44 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.997507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196f48-f917-4882-91d3-4a96b90e4d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.997786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196f48-f917-4882-91d3-4a96b90e4d44 None admin] PATH_INFO: `/v3/projects/dfc069bb73144aedb17c6ed3f5373d90/users/8e960b70ef1f420582557a38b71c2904/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.999751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b196f48-f917-4882-91d3-4a96b90e4d44 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc069bb73144aedb17c6ed3f5373d90, user_id=8e960b70ef1f420582557a38b71c2904, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:53.001188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b196f48-f917-4882-91d3-4a96b90e4d44 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:53.078139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 612/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:43:52 2022] PUT /identity/v3/projects/dfc069bb73144aedb17c6ed3f5373d90/users/8e960b70ef1f420582557a38b71c2904/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:55.041851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12f9c4-9136-4192-acd2-353f9a21eaa0 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.042372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12f9c4-9136-4192-acd2-353f9a21eaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.042372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12f9c4-9136-4192-acd2-353f9a21eaa0 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.043701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 581/1193] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:55.048743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b2b2-d8ba-406b-a1a8-cb5d78acd76b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.049123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b2b2-d8ba-406b-a1a8-cb5d78acd76b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.049306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b2b2-d8ba-406b-a1a8-cb5d78acd76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.068139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9d69b2b2-d8ba-406b-a1a8-cb5d78acd76b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:55.069320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9d69b2b2-d8ba-406b-a1a8-cb5d78acd76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:55.071243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:55.134170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 613/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:43:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:43:55.156956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12f9c4-9136-4192-acd2-353f9a21eaa0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.172046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:55.258750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12f9c4-9136-4192-acd2-353f9a21eaa0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:55.290388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:55.291016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.291294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.291593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.302418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:43:55.307243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:43:55.308421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:43:55.309327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 582/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:43:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:43:55.314399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69b2b2-d8ba-406b-a1a8-cb5d78acd76b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.411504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69b2b2-d8ba-406b-a1a8-cb5d78acd76b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:55.444520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:55.445131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.445416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.445692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.446411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:55.446721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:55.448132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:55.452989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:55.456266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 614/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:43:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:55.462787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.531179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:55.559990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761299bb-2323-4a35-ac99-dca85b78222c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:55.593799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:55.594433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.594718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.594995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] PATH_INFO: `/v3/users/8e960b70ef1f420582557a38b71c2904` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.596169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] RBAC: Authorizing `identity:get_user(user_id=8e960b70ef1f420582557a38b71c2904)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:55.597743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:55.599343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 583/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 07:43:55 2022] GET /identity/v3/users/8e960b70ef1f420582557a38b71c2904 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:55.605045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.702940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c24f5d-c385-4167-be33-09a2ce81d41b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:55.736405 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:55.737013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.737298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.737574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.803742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:55.805929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:55.807391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:55.905843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 615/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:43:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:43:55.911986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.010425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45b875-e6b0-4d66-9e4d-d5f5e0c8349a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.043122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1374f6-1108-4bc8-a91a-996cea0c9227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.043775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1374f6-1108-4bc8-a91a-996cea0c9227 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.044056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1374f6-1108-4bc8-a91a-996cea0c9227 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.044327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1374f6-1108-4bc8-a91a-996cea0c9227 None admin] PATH_INFO: `/v3/projects/dfc069bb73144aedb17c6ed3f5373d90` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.045466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1374f6-1108-4bc8-a91a-996cea0c9227 None admin] RBAC: Authorizing `identity:get_project(project_id=dfc069bb73144aedb17c6ed3f5373d90)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.046892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1374f6-1108-4bc8-a91a-996cea0c9227 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.048556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 584/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 07:43:55 2022] GET /identity/v3/projects/dfc069bb73144aedb17c6ed3f5373d90 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:56.054608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.149457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:56.152365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384afc9-bb7f-41b9-a719-b5609ddb9ed5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.185216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e94e4-4269-416a-b52f-abe9f3956639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.185767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e94e4-4269-416a-b52f-abe9f3956639 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.186054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e94e4-4269-416a-b52f-abe9f3956639 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.186331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e94e4-4269-416a-b52f-abe9f3956639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.187119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e94e4-4269-416a-b52f-abe9f3956639 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=dfc069bb73144aedb17c6ed3f5373d90, user.id=8e960b70ef1f420582557a38b71c2904) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:56.187433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e94e4-4269-416a-b52f-abe9f3956639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.188894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e94e4-4269-416a-b52f-abe9f3956639 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.195155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 616/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:43:56 2022] GET /identity/v3/role_assignments?user.id=8e960b70ef1f420582557a38b71c2904&scope.project.id=dfc069bb73144aedb17c6ed3f5373d90&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:44:06.039159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:06.192630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=18 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:07.877326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e55c4e-30fb-459c-9606-ae699b2846dc None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:07.877906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e55c4e-30fb-459c-9606-ae699b2846dc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:07.877906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e55c4e-30fb-459c-9606-ae699b2846dc None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:07.879271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 585/1201] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:07.884337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71699e9f-2ac0-453f-a401-ffa72ccdf539 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:07.884789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71699e9f-2ac0-453f-a401-ffa72ccdf539 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:07.884889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71699e9f-2ac0-453f-a401-ffa72ccdf539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:07.902600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-71699e9f-2ac0-453f-a401-ffa72ccdf539 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:07.903840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-71699e9f-2ac0-453f-a401-ffa72ccdf539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:07.970248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 617/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:07.994488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e55c4e-30fb-459c-9606-ae699b2846dc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:08.097218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e55c4e-30fb-459c-9606-ae699b2846dc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:08.133031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:08.133669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:08.133953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:08.134229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:08.144453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:08.145891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:08.160962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:08.168958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:08.170960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:08.170960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:08.170960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:08.170960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:08.170960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 586/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:08.171348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71699e9f-2ac0-453f-a401-ffa72ccdf539 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:08.269901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71699e9f-2ac0-453f-a401-ffa72ccdf539 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:08.302469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:08.308193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15afd84-fb8d-406a-a31f-0171b6993da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:08.308809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15afd84-fb8d-406a-a31f-0171b6993da1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:08.309090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15afd84-fb8d-406a-a31f-0171b6993da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:08.309362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15afd84-fb8d-406a-a31f-0171b6993da1 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:08.310227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15afd84-fb8d-406a-a31f-0171b6993da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:08.310794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15afd84-fb8d-406a-a31f-0171b6993da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:08.312186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15afd84-fb8d-406a-a31f-0171b6993da1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:08.323130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 618/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:08.329230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:08.383000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:08.429411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab47e6-e122-4f72-8bd5-81ef5d7b710b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:08.463181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cce1f7-d675-46be-983b-f74ac5f0a9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:08.463829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cce1f7-d675-46be-983b-f74ac5f0a9df None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:08.464117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cce1f7-d675-46be-983b-f74ac5f0a9df None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:08.464409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cce1f7-d675-46be-983b-f74ac5f0a9df None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:08.465097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cce1f7-d675-46be-983b-f74ac5f0a9df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:08.467406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cce1f7-d675-46be-983b-f74ac5f0a9df None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:08.617173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 587/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 07:45:08 2022] POST /identity/v3/users => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:10.661813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7092c-6f76-4fa8-a939-df8c0472bfef None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:10.662381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7092c-6f76-4fa8-a939-df8c0472bfef None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:10.662381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7092c-6f76-4fa8-a939-df8c0472bfef None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:10.663807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 619/1206] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:10.669188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89e44d-5eb2-4b25-b4b9-ae155a6b7180 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:10.669596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89e44d-5eb2-4b25-b4b9-ae155a6b7180 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:10.669739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89e44d-5eb2-4b25-b4b9-ae155a6b7180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:10.688893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7b89e44d-5eb2-4b25-b4b9-ae155a6b7180 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:10.690079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7b89e44d-5eb2-4b25-b4b9-ae155a6b7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:10.758369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 588/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:45:10.780397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7092c-6f76-4fa8-a939-df8c0472bfef None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:10.883456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7092c-6f76-4fa8-a939-df8c0472bfef None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:10.917112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13105100-804f-4f69-9802-260fd5213175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:10.917686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13105100-804f-4f69-9802-260fd5213175 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:10.917972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13105100-804f-4f69-9802-260fd5213175 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:10.918251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13105100-804f-4f69-9802-260fd5213175 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:10.929823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13105100-804f-4f69-9802-260fd5213175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-13105100-804f-4f69-9802-260fd5213175 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:45:10.935915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:10.937106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:10.938153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 620/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:45:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:10.942741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89e44d-5eb2-4b25-b4b9-ae155a6b7180 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:10.982152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:11.039746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89e44d-5eb2-4b25-b4b9-ae155a6b7180 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.074409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.075006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.075285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.075574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.076532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:11.076835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.078250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.085909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 589/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:45:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:11.092292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13105100-804f-4f69-9802-260fd5213175 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.142491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:11.193462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13105100-804f-4f69-9802-260fd5213175 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.226891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.227548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.227845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.228125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.239070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.245901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:11.254603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:11.255714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:11.256881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:11.256881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:11.256881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:11.256881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:11.256881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 621/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:11.261800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.358037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:11.360910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2e3d-cad4-4615-9171-d6f02ff37190 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.392850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.393448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.393731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.394002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.394685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:11.394991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.396382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.407756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 590/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:11.413399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.512040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006f35f-126f-4312-a596-871bd7f6acf7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.514085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:11.545671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.546307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.546593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.546867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.554122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.555591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:45:11.566839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:11.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:11.569241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 622/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:11.573897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.674953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef93a38-8337-4235-9094-60888fc75f52 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.708010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.708630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.708931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.709196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.709980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:11.710294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.711741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.729104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 591/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 07:45:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:11.735055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.836867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568379a-ec25-43ba-aef7-34a87716ad28 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.870500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.871124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.871413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.871766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.902713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:11.940002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.941463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.040641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 623/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:45:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:45:12.046937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.066454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:12.150309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00313a14-214a-447f-82fc-e77dceb24607 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.184088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.185258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.185631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.185873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.196140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.197588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:12.214141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:12.215078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:12.216757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:12.216757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:12.216757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:12.216757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:12.216757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 592/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:12.221566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.281964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:12.320256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100cc3d-a57d-4482-82a5-6d3f062b8d33 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.355409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.356052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.356348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.356827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.357782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:12.358087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.359457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.370453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 624/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:12.376273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.443957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:12.475788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2379f-1fd5-4eb8-bc99-0c37651960bf None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.508758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.509395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.509681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.509954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.517048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.518538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:45:12.530446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:12.531510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:12.532796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 593/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:12.537471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.638356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb857ad-77ee-4929-ae27-f9f1860ea81d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.659522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:12.670807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3b38-24e7-4ee7-8195-ecc150eb3435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.671428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3b38-24e7-4ee7-8195-ecc150eb3435 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.671765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3b38-24e7-4ee7-8195-ecc150eb3435 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.672011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3b38-24e7-4ee7-8195-ecc150eb3435 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.672755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3b38-24e7-4ee7-8195-ecc150eb3435 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:12.673063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3b38-24e7-4ee7-8195-ecc150eb3435 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.674502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3b38-24e7-4ee7-8195-ecc150eb3435 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.686543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 625/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:12.692779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.793428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074f5cc-a385-46c1-910c-304bfbc6f255 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.820894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:12.827490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1804ff-5cfd-46c9-9b8c-dc9c6a70794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.828129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1804ff-5cfd-46c9-9b8c-dc9c6a70794a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.828410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1804ff-5cfd-46c9-9b8c-dc9c6a70794a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.828682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1804ff-5cfd-46c9-9b8c-dc9c6a70794a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.829461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1804ff-5cfd-46c9-9b8c-dc9c6a70794a None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=5b26619beabb4e8f96d9f08ecb107027) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:12.829767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1804ff-5cfd-46c9-9b8c-dc9c6a70794a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.831178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1804ff-5cfd-46c9-9b8c-dc9c6a70794a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.836610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 594/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/role_assignments?user.id=5b26619beabb4e8f96d9f08ecb107027&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:14.805719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d296ea-4361-46b6-91fc-9457b1a4880f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:14.806215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d296ea-4361-46b6-91fc-9457b1a4880f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:14.806315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d296ea-4361-46b6-91fc-9457b1a4880f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:14.807650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 626/1220] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:14.813663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299febc-e259-4955-94af-049e0d11e8c2 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:14.814114 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299febc-e259-4955-94af-049e0d11e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:14.814366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299febc-e259-4955-94af-049e0d11e8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:14.833166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8299febc-e259-4955-94af-049e0d11e8c2 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:14.834389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8299febc-e259-4955-94af-049e0d11e8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:14.901583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 595/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:14.925465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d296ea-4361-46b6-91fc-9457b1a4880f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.026920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d296ea-4361-46b6-91fc-9457b1a4880f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.061965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.062611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.062896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.063172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.073249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:45:15.078628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:15.079831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:15.088345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 627/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:45:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:15.089216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299febc-e259-4955-94af-049e0d11e8c2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.187884 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299febc-e259-4955-94af-049e0d11e8c2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.211553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:15.222922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.223568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.223858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.224133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.224855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:15.225160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.226545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.235188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 596/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:15.240952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.286318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:15.340195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeb87a-1f97-487e-b8d4-79aec9801672 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.373497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.374128 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.374408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.374677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.384628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.386078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:15.401145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:15.402171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:15.404008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:15.404008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:15.404008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:15.404008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:15.404008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 628/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:15.408827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.514124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a9697-37f7-49e1-ae6c-f96f5add8456 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.547803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.548378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.548659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.548932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.549649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:15.549992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.551369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.563599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 597/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:15.569297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.664048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:15.670511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb47ac-fdd3-4ea0-83c8-2feb0170d037 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.704164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.704799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.705083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.705357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.712734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.714205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:45:15.725259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:15.726384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:15.727600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 629/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:15.732812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.782946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:15.836859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97959a-dd8c-4735-b78e-2d5570993d72 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.873307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.873928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.874213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.874489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.875277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:15.875622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.877047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.895949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 598/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:45:15.901818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.000388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31dc9-0913-4a25-882d-a8065a977977 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.032877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.033512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.033797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.034071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.044968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.046420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:16.061097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:16.062012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:16.063053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:16.063053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:16.063053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:16.063053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:16.063053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 630/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:16.068754 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.169876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc790e-4fbd-4b01-b48d-dab1f7a69c51 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.172187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:16.203604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.204220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.204504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.204778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.205459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:16.205765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.207128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.218135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 599/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:16.224518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.234228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:16.324438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5409cb-d6f8-4229-84fd-3aa919f3fe3e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.358793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.359438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.359765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.360025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.366424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.367901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:45:16.379243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:16.380354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:16.381625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 631/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:16.386362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.487471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591ec6c-d0c7-4e37-bca4-7d23400425c8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.520667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7ddb1-dd38-41b7-9752-897881c697f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.521313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7ddb1-dd38-41b7-9752-897881c697f4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.521596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7ddb1-dd38-41b7-9752-897881c697f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.521866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7ddb1-dd38-41b7-9752-897881c697f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.522621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7ddb1-dd38-41b7-9752-897881c697f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:16.522924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7ddb1-dd38-41b7-9752-897881c697f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.524370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7ddb1-dd38-41b7-9752-897881c697f4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.536509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 600/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:16.542461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.606887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:16.641167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1036e-7e78-4c9e-a26d-691c39edb6fe None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.674800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c268c9f-4aa8-4f1a-97cb-042fc18a06a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.675518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c268c9f-4aa8-4f1a-97cb-042fc18a06a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.675830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c268c9f-4aa8-4f1a-97cb-042fc18a06a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.676113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c268c9f-4aa8-4f1a-97cb-042fc18a06a6 None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/5b26619beabb4e8f96d9f08ecb107027/roles/49afe9a7e8ab49dca85e0aa074a1ee33` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.691228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c268c9f-4aa8-4f1a-97cb-042fc18a06a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=5b26619beabb4e8f96d9f08ecb107027, role_id=49afe9a7e8ab49dca85e0aa074a1ee33)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.692714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c268c9f-4aa8-4f1a-97cb-042fc18a06a6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.821757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 632/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:45:16 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/5b26619beabb4e8f96d9f08ecb107027/roles/49afe9a7e8ab49dca85e0aa074a1ee33 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:18.790331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bd3d1-812d-469f-94e7-8c361775734c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:18.790850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bd3d1-812d-469f-94e7-8c361775734c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:18.790953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bd3d1-812d-469f-94e7-8c361775734c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:18.792364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 601/1233] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:18.797632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902425b-9703-4fe0-918e-718feb45007d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:18.798032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902425b-9703-4fe0-918e-718feb45007d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:18.798203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902425b-9703-4fe0-918e-718feb45007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:18.816474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c902425b-9703-4fe0-918e-718feb45007d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:18.817680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c902425b-9703-4fe0-918e-718feb45007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:18.888989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 633/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:18.911955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bd3d1-812d-469f-94e7-8c361775734c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:18.937455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:19.012885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bd3d1-812d-469f-94e7-8c361775734c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.046633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.047384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.047782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.048064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.058036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:45:19.063740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:19.064897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:19.065888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 602/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 07:45:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:19.071093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902425b-9703-4fe0-918e-718feb45007d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.117462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:19.170781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902425b-9703-4fe0-918e-718feb45007d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.205839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.206458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.206741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.207015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.207759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:19.208063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:19.209557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:19.218096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 634/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:19.223944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.310803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:19.326121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1120a-928d-49bd-8e22-76ce3b34e4f0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.359793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.360431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.360714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.360986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.370597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:19.372283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:19.386421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:19.387880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:19.390654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:19.390654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:19.390654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:19.390654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:19.390654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 603/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:19.393361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.492973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819064a-3459-42a0-8225-142f7edc0002 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.501658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:19.525419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.526030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.526316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.526592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.527276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:19.528189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:19.529690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:19.541964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 635/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:19.548504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.645341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357bb1c-635c-481e-845a-0682f112d8c1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.678230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.678856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.679138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.679414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] PATH_INFO: `/v3/users/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.682068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:19.687121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:19.688586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:45:19.701209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:19.709176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:19.710252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 604/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:19.710888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.808028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da322de6-b317-4f80-96a8-b4d888a09eac None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.843360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.844033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.844318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.844593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.845491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:19.845805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:19.847237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:19.864125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 636/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:19.869960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.970522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c78e0b-1b22-497b-b11e-48fc9fd94cba None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:20.003396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:20.004038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:20.004326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:20.004603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:20.064696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:20.071112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:20.072596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:20.168608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 605/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:45:20.174468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:20.202392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:20.273136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48528ed5-0341-417d-92ee-7faf35dfff8d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:20.306347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:20.306983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:20.307267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:20.307562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:20.317859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:20.319300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:20.334769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:20.335848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:20.336958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:20.336958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:20.336958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:20.336958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:20.336958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 637/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:20.341846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:20.433761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:20.440068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e557c1-a780-43da-9542-c9d57ed5d045 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:20.474249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:20.474860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:20.475146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:20.475421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:20.476145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:20.476474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:20.477858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:20.489945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 606/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:45:20.495780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:20.578890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:20.599122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939e24d-2820-458b-a354-46e8eef474a4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:20.632414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:20.633053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:20.633339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:20.633618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:20.640325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:20.641793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:45:20.654908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:20.655985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:20.661771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 638/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:45:20.662279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:20.765636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5959cf-cfe0-45c4-8a43-8c8e2f235d41 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:20.799918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15aa361-3bc0-46da-8aa2-45bfbde766e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:20.800542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15aa361-3bc0-46da-8aa2-45bfbde766e2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:20.800888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15aa361-3bc0-46da-8aa2-45bfbde766e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:20.801173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15aa361-3bc0-46da-8aa2-45bfbde766e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:20.801922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15aa361-3bc0-46da-8aa2-45bfbde766e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:20.802234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15aa361-3bc0-46da-8aa2-45bfbde766e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:20.803695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15aa361-3bc0-46da-8aa2-45bfbde766e2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:20.815642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 607/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:20.822015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:20.922149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18d082-3d01-474f-8162-8f3c240cf13f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:20.956144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf24c26-9609-4b73-a569-eef45e59d2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:20.956753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24c26-9609-4b73-a569-eef45e59d2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:20.957036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24c26-9609-4b73-a569-eef45e59d2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:20.957337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24c26-9609-4b73-a569-eef45e59d2c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:20.958084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf24c26-9609-4b73-a569-eef45e59d2c9 None admin] RBAC: Adding query filter params (role.id=49afe9a7e8ab49dca85e0aa074a1ee33, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=5b26619beabb4e8f96d9f08ecb107027) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:20.958388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf24c26-9609-4b73-a569-eef45e59d2c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:20.959863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf24c26-9609-4b73-a569-eef45e59d2c9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:20.965935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 639/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:45:20 2022] GET /identity/v3/role_assignments?user.id=5b26619beabb4e8f96d9f08ecb107027&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=49afe9a7e8ab49dca85e0aa074a1ee33 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:23.040679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acde3-59d7-40d7-830c-210b2c913bc1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:23.041182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acde3-59d7-40d7-830c-210b2c913bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:23.041182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acde3-59d7-40d7-830c-210b2c913bc1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:23.042492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 608/1247] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:23.048232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8bca8-8f22-4ebe-8875-3e16be1ed4cd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:23.048626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8bca8-8f22-4ebe-8875-3e16be1ed4cd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:23.048804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8bca8-8f22-4ebe-8875-3e16be1ed4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:23.055170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:23.067586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-67f8bca8-8f22-4ebe-8875-3e16be1ed4cd None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:23.068788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67f8bca8-8f22-4ebe-8875-3e16be1ed4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:23.135437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 640/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:23.159748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acde3-59d7-40d7-830c-210b2c913bc1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:23.178106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:23.263876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acde3-59d7-40d7-830c-210b2c913bc1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:23.299812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:23.300389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:23.300676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:23.300955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:23.311154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:45:23.316042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:23.317165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:23.318136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 609/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:45:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:23.323808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8bca8-8f22-4ebe-8875-3e16be1ed4cd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:23.425108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8bca8-8f22-4ebe-8875-3e16be1ed4cd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:23.455188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:23.460027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:23.460651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:23.460939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:23.461218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:23.461941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:23.462249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:23.463651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:23.472477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 641/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:45:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:45:23.478128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:23.554578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:23.577713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5adb6-9856-4fa5-816e-d1fde886471f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:23.611504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:23.612164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:23.612443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:23.612714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:23.622979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:23.624443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:23.639825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:23.641029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:23.641948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:23.641948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:23.641948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:23.641948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:23.641948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 610/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:23.646775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:23.747478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4955e5c-16d2-4a7a-bede-acc47706932d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:23.783209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:23.783877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:23.784166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:23.784445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:23.785168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:23.785472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:23.786850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:23.798576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 642/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:23.804383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:23.905344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf62529-ce1b-46be-b3b2-5278e1118c02 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:23.931360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:23.939770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:23.940419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:23.940707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:23.940987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:23.949840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:23.951583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:45:23.962887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:23.964036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:23.965476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 611/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:23.970170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:24.026087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:24.067521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdb5a0-779a-45b6-a97c-00190a49aa59 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:24.100940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:24.106386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:24.106386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:24.106386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:24.106386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:24.106386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:24.106386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:24.122536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 643/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 07:45:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:24.128452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:24.231440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17788903-49b0-4fad-9ef0-4bceb9585a1a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:24.265262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:24.265872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:24.266151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:24.266421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:24.320561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:24.332914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:24.341588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:24.431824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 612/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:45:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:45:24.438675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:24.536432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51881f4-3f30-45bd-9e40-4187b1fe1966 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:24.550010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:24.571015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:24.571677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:24.571961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:24.572235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:24.582475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:24.583953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:24.598735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:24.599915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:24.601245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:24.601245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:24.601245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:24.601245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:24.601245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 644/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:24.606470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:24.691998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:24.703947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1264f73-de6d-490c-95a7-c69319e1dee8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:24.737428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:24.738032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:24.738312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:24.738581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:24.739260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:24.739584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:24.740951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:24.752998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 613/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:45:24.757939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:24.858982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956762c9-2711-42d3-b185-55e2494aada7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:24.891413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:24.892072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:24.892356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:24.892629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:24.899095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:24.900578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:45:24.912767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:24.913836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:24.914820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 645/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:45:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:24.920505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:25.016974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37383306-7b52-496c-90a5-b1d17f9cb126 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:25.049356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9a5df-0f1f-4a6d-b259-1943c953bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:25.049955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9a5df-0f1f-4a6d-b259-1943c953bb25 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:25.050236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9a5df-0f1f-4a6d-b259-1943c953bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:25.050509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9a5df-0f1f-4a6d-b259-1943c953bb25 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:25.051245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9a5df-0f1f-4a6d-b259-1943c953bb25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:25.051569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9a5df-0f1f-4a6d-b259-1943c953bb25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:25.052999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9a5df-0f1f-4a6d-b259-1943c953bb25 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:25.064768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 614/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:25.070794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:25.097591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:25.170361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a8a75-ee36-41b4-9dd3-38f2d24b7ee6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:25.204077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe08e13-e4ae-4d3c-baf6-444ddccc4ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:25.204692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe08e13-e4ae-4d3c-baf6-444ddccc4ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:25.204977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe08e13-e4ae-4d3c-baf6-444ddccc4ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:25.205251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe08e13-e4ae-4d3c-baf6-444ddccc4ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:25.206036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe08e13-e4ae-4d3c-baf6-444ddccc4ec0 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=5b26619beabb4e8f96d9f08ecb107027) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:25.206433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe08e13-e4ae-4d3c-baf6-444ddccc4ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:25.207932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe08e13-e4ae-4d3c-baf6-444ddccc4ec0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:25.213300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 646/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:45:25 2022] GET /identity/v3/role_assignments?user.id=5b26619beabb4e8f96d9f08ecb107027&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=05421dac98a848f68d13ca21036dbfbb => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:45:27.196204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65e522-eb34-42b2-a698-52015f1da540 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.196759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65e522-eb34-42b2-a698-52015f1da540 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.196861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65e522-eb34-42b2-a698-52015f1da540 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.198282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 615/1261] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:27.204453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17fec2-fcb5-4dbc-b570-124f4eeaee84 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.204888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17fec2-fcb5-4dbc-b570-124f4eeaee84 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.204994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17fec2-fcb5-4dbc-b570-124f4eeaee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.223946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2f17fec2-fcb5-4dbc-b570-124f4eeaee84 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:27.225188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2f17fec2-fcb5-4dbc-b570-124f4eeaee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:27.294686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 647/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:27.316662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65e522-eb34-42b2-a698-52015f1da540 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.327705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:27.415123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65e522-eb34-42b2-a698-52015f1da540 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.449925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.450765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.451051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.451329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.462604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:45:27.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:27.468958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:27.469824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 616/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:27.475175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17fec2-fcb5-4dbc-b570-124f4eeaee84 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.494276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:27.573406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17fec2-fcb5-4dbc-b570-124f4eeaee84 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.608087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.608701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.608991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.609263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.609971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:27.610277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:27.611680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:27.619429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 648/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:27.625539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.693525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:27.727331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695f9f5-be52-443c-9985-566bb6976c42 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.761508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.762136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.762415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.762694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.773753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:27.775280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:27.789980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:27.795839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:27.796956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:27.796956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:27.796956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:27.796956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:27.796956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 617/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:27.798046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.879791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:27.898398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78b7b0-97cd-485c-a24b-1c21391b8f26 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.931607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.932228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.932517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.932795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.933799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:27.933972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:27.935411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:27.946740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 649/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:45:27.953402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.053543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90f5c6-95dd-4388-befd-029bf35d109d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.072806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:28.087106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.087783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.088071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.088348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] PATH_INFO: `/v3/users/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.097143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.100621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:45:28.110001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:28.111157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:28.112489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 618/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:28.117290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.218105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5e3b-5d77-4b37-981d-70d3b58fd5f2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.252147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.252764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.253043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.253341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.254124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:28.254429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.255888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.265313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:28.274927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 650/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 07:45:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:28.281955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.382810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07974-da43-4b13-a97e-2c1ed2735fef None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.417173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.418031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.418318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.418595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.429491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.431048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:28.446461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:28.447591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:28.448805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:28.448805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:28.448805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:28.448805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:28.448805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 619/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:28.454329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.553025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dda9d7-fc6c-4364-bef5-ba3710438c52 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.587442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.588089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.588376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.588688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.589411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:28.589778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.591160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.602237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 651/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:28.608461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.641417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:28.711417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f1c4c-b748-4964-82d4-dbce69958a49 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.748139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.748780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.749066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.749343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.758779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.760262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:45:28.772906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:28.777464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:28.778530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 620/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:45:28.780148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.838912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:28.880026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65ac4-fed6-4c3c-83b7-30d61548b8c5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.913382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb33051-f947-4c8b-8ecb-8817af1e7f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.913944 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb33051-f947-4c8b-8ecb-8817af1e7f54 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.914231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb33051-f947-4c8b-8ecb-8817af1e7f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.914509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb33051-f947-4c8b-8ecb-8817af1e7f54 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.915259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb33051-f947-4c8b-8ecb-8817af1e7f54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:28.915600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb33051-f947-4c8b-8ecb-8817af1e7f54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.917133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb33051-f947-4c8b-8ecb-8817af1e7f54 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.929495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 652/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:28.936354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.026170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:29.034621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd56ae0-ae03-47d0-991a-84360913d225 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.068056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06a2af-cf7a-4619-873b-2e1dfcb67f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.068719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06a2af-cf7a-4619-873b-2e1dfcb67f59 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.069001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06a2af-cf7a-4619-873b-2e1dfcb67f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.069280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06a2af-cf7a-4619-873b-2e1dfcb67f59 None admin] PATH_INFO: `/v3/projects/d4b176962cc74ab88204a4271aad8278/users/5b26619beabb4e8f96d9f08ecb107027/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.071183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06a2af-cf7a-4619-873b-2e1dfcb67f59 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b176962cc74ab88204a4271aad8278, user_id=5b26619beabb4e8f96d9f08ecb107027, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.072648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06a2af-cf7a-4619-873b-2e1dfcb67f59 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.273119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 621/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 07:45:28 2022] PUT /identity/v3/projects/d4b176962cc74ab88204a4271aad8278/users/5b26619beabb4e8f96d9f08ecb107027/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:31.296797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c348-de2f-4cd0-af56-c15a96c58f0d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:31.297326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c348-de2f-4cd0-af56-c15a96c58f0d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:31.297326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c348-de2f-4cd0-af56-c15a96c58f0d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:31.298698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 653/1274] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:31.303817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4521c-d4f3-4860-ada6-3db986ee0551 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:31.304247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4521c-d4f3-4860-ada6-3db986ee0551 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:31.304397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4521c-d4f3-4860-ada6-3db986ee0551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:31.323058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-65a4521c-d4f3-4860-ada6-3db986ee0551 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:31.324442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-65a4521c-d4f3-4860-ada6-3db986ee0551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:31.389647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 622/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:31.412340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c348-de2f-4cd0-af56-c15a96c58f0d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:31.514464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c348-de2f-4cd0-af56-c15a96c58f0d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:31.531739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:31.549103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:31.549674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:31.549957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:31.550230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:31.560490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:45:31.565809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:31.566972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:31.568097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 654/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 07:45:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:31.573284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4521c-d4f3-4860-ada6-3db986ee0551 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:31.652717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:31.670219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4521c-d4f3-4860-ada6-3db986ee0551 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:31.702601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:31.703210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:31.703486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:31.703789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:31.704490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:31.704791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:31.706138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:31.713926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 623/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 07:45:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:31.720365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:31.817681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69988f4f-0196-428c-ac94-10a171ea9fc5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:31.851466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:31.852134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:31.852419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:31.852693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:31.863112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:31.864636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:31.878443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:31.881136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:31.882112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:31.882112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:31.882112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:31.882112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:31.882112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 655/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:31.885502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:31.986675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325732d-f7bf-457e-b1c6-538cfde70333 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:32.019984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:32.020590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:32.020868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:32.021140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:32.021821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:32.022124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:32.023480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:32.035910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 624/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:45:32.041570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:32.072497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:32.143891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed9f3e-a5f6-442c-a6fa-d8b76ec0563a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:32.177640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:32.178275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:32.178554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:32.178829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:32.186832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:32.188302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:45:32.200681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:32.201871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:32.203073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 656/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:45:32.207831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:32.211630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:32.304725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd71b2d-3686-4e3f-8f5f-1e36c6de5e2f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:32.339251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:32.339822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:32.340104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:32.340375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:32.341151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:32.341453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:32.342878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:32.359861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 625/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 07:45:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:32.365904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:32.458721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:32.468086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07e32-582f-45d5-823f-5c7b39a78542 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:32.501366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:32.501970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:32.502250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:32.502521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:32.568963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:32.570431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:32.658551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:32.669342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 657/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 07:45:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:45:32.675366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:32.737264 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:32.776323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274fd73-79f2-412e-b2da-15ad3911a4b4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:32.810490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:32.811120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:32.811402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:32.811707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:32.822097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:32.823772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:45:32.838807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:32.840361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:32.841314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:45:32.841314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:45:32.841314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:45:32.841314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:32.841314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 626/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:32.846546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:32.945008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475047ba-9af5-4329-bed1-bef9222376ff None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:32.978639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:32.979355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:32.980322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:32.980672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:32.981326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:32.981637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:32.983023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:32.994651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 658/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 07:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:33.000436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:33.098282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e4263-f7df-4a46-90c6-ee7263d136fa None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:33.112214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:33.132068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:33.132728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:33.133013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:33.133288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:33.140011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:33.141440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:45:33.153056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:33.154116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Aug 01 07:45:33.155445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 627/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 07:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:33.160569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:33.209986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:33.258263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7db21-ea12-4e65-afc6-ea8cf87c6b9d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:33.292577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee710c-8ba8-4750-b1db-2e2fe912060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:33.293249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee710c-8ba8-4750-b1db-2e2fe912060d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:33.293537 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee710c-8ba8-4750-b1db-2e2fe912060d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:33.293815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee710c-8ba8-4750-b1db-2e2fe912060d None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:33.294563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee710c-8ba8-4750-b1db-2e2fe912060d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:33.294874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee710c-8ba8-4750-b1db-2e2fe912060d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:33.296345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee710c-8ba8-4750-b1db-2e2fe912060d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:33.308599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 659/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 07:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:33.314771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:33.414276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f2f19-a63a-4d41-8616-67345c438c70 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:33.447216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b548ca-7890-4cf1-a6c8-ec07d9554d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:33.447935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b548ca-7890-4cf1-a6c8-ec07d9554d35 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:33.448726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b548ca-7890-4cf1-a6c8-ec07d9554d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:33.449003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b548ca-7890-4cf1-a6c8-ec07d9554d35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:33.450029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b548ca-7890-4cf1-a6c8-ec07d9554d35 None admin] RBAC: Adding query filter params (role.id=05421dac98a848f68d13ca21036dbfbb, scope.project.id=d4b176962cc74ab88204a4271aad8278, user.id=5b26619beabb4e8f96d9f08ecb107027) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:33.450336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b548ca-7890-4cf1-a6c8-ec07d9554d35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:33.451775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b548ca-7890-4cf1-a6c8-ec07d9554d35 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:33.457522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 628/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 07:45:33 2022] GET /identity/v3/role_assignments?user.id=5b26619beabb4e8f96d9f08ecb107027&scope.project.id=d4b176962cc74ab88204a4271aad8278&role.id=05421dac98a848f68d13ca21036dbfbb => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:35.506558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955ed6-8a8e-4ecf-b591-f5781c8d2f21 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:35.507089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955ed6-8a8e-4ecf-b591-f5781c8d2f21 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:35.507182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955ed6-8a8e-4ecf-b591-f5781c8d2f21 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:35.508511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 660/1288] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:35.513851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f39741-7fc3-485d-ad2a-8299d6fae2bb None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:35.514286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f39741-7fc3-485d-ad2a-8299d6fae2bb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:35.514415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f39741-7fc3-485d-ad2a-8299d6fae2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:35.532935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-00f39741-7fc3-485d-ad2a-8299d6fae2bb None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:35.534171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-00f39741-7fc3-485d-ad2a-8299d6fae2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:35.565873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:35.602910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 629/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:35.626331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e955ed6-8a8e-4ecf-b591-f5781c8d2f21 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:35.725367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e955ed6-8a8e-4ecf-b591-f5781c8d2f21 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:35.736399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:35.759650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:35.760407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:35.760573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:35.760943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:35.761570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:35.762946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:35.771894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:35.773143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:35.774032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 661/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:45:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:35.780162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f39741-7fc3-485d-ad2a-8299d6fae2bb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:35.879494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f39741-7fc3-485d-ad2a-8299d6fae2bb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:35.912750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a733e-2dd4-4d2e-8739-2235ea869aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:35.913360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a733e-2dd4-4d2e-8739-2235ea869aac None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:35.913678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a733e-2dd4-4d2e-8739-2235ea869aac None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:35.913957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a733e-2dd4-4d2e-8739-2235ea869aac None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:35.914631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a733e-2dd4-4d2e-8739-2235ea869aac None admin] RBAC: Adding query filter params (name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:35.914939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a733e-2dd4-4d2e-8739-2235ea869aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:35.916511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a733e-2dd4-4d2e-8739-2235ea869aac None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:35.921979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 630/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 07:45:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:35.928151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:36.026695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19d3d0-91d3-4c4f-aa1f-866994d20fc4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:36.059791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711677b6-4204-45bf-ade3-0f9f9b3b8a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:36.060397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711677b6-4204-45bf-ade3-0f9f9b3b8a9a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:36.060678 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711677b6-4204-45bf-ade3-0f9f9b3b8a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:36.060960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711677b6-4204-45bf-ade3-0f9f9b3b8a9a None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:36.061625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711677b6-4204-45bf-ade3-0f9f9b3b8a9a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:36.061928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711677b6-4204-45bf-ade3-0f9f9b3b8a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:36.063292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711677b6-4204-45bf-ade3-0f9f9b3b8a9a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:36.067650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 662/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 07:45:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:38.072349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05345a6a-6123-4295-99cc-bd4264915bed None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:38.073387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05345a6a-6123-4295-99cc-bd4264915bed None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:38.073387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05345a6a-6123-4295-99cc-bd4264915bed None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:38.075162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 631/1293] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:38.080273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070757-7643-4792-b2eb-a0585221fbf6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:38.080695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070757-7643-4792-b2eb-a0585221fbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:38.080829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070757-7643-4792-b2eb-a0585221fbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:38.099320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-52070757-7643-4792-b2eb-a0585221fbf6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:38.100522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-52070757-7643-4792-b2eb-a0585221fbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:38.129173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:38.165214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 663/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:45:38.188376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05345a6a-6123-4295-99cc-bd4264915bed None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:38.220750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:38.290794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05345a6a-6123-4295-99cc-bd4264915bed None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:38.325976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce6122-7672-487e-bf15-def82e5cf2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:38.326528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce6122-7672-487e-bf15-def82e5cf2ea None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:38.326812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce6122-7672-487e-bf15-def82e5cf2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:38.327083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce6122-7672-487e-bf15-def82e5cf2ea None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:38.327794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce6122-7672-487e-bf15-def82e5cf2ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:38.329162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce6122-7672-487e-bf15-def82e5cf2ea None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:38.439120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 632/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 07:45:38 2022] POST /identity/v3/services => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:40.440198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d27a1-158b-48cb-a716-d0ef062b53f2 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:40.440755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d27a1-158b-48cb-a716-d0ef062b53f2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:40.440755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d27a1-158b-48cb-a716-d0ef062b53f2 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:40.442122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 664/1296] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:40.447137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4806d-02ff-49cd-832c-260bb2583118 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:40.447587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4806d-02ff-49cd-832c-260bb2583118 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:40.448293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4806d-02ff-49cd-832c-260bb2583118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:40.466864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-19b4806d-02ff-49cd-832c-260bb2583118 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:40.468104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-19b4806d-02ff-49cd-832c-260bb2583118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:40.539918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 633/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:40.562184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d27a1-158b-48cb-a716-d0ef062b53f2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:40.664906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d27a1-158b-48cb-a716-d0ef062b53f2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:40.699811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:40.700384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:40.700666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:40.700943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:40.701606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:40.702988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:40.711920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:40.712979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:40.714023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 665/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:45:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:40.719350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4806d-02ff-49cd-832c-260bb2583118 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:40.738922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:40.819795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4806d-02ff-49cd-832c-260bb2583118 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:40.854296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:40.854911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:40.855193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:40.855463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:40.856162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:40.856467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:40.857820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:40.862891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 634/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 07:45:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:40.869126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:40.901838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:40.970958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528e272-1324-44bc-9282-877f3a115944 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:41.003215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41d32f-22b1-4f31-ae30-78294e217c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:41.003983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41d32f-22b1-4f31-ae30-78294e217c23 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:41.004337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41d32f-22b1-4f31-ae30-78294e217c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:41.004719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41d32f-22b1-4f31-ae30-78294e217c23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:41.005480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41d32f-22b1-4f31-ae30-78294e217c23 None admin] RBAC: Adding query filter params (interface=public, service_id=f3eb2dc7d6cf422c8d63e5d11dbde9b0, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:41.005800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41d32f-22b1-4f31-ae30-78294e217c23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:41.007194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41d32f-22b1-4f31-ae30-78294e217c23 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:41.012988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 666/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 07:45:40 2022] GET /identity/v3/endpoints?service_id=f3eb2dc7d6cf422c8d63e5d11dbde9b0&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:41.018732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:41.099787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:41.117119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb247372-79a6-4d12-b935-140b45210924 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:41.150520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ec55-e541-4ca2-a728-89b72b099a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:41.151159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ec55-e541-4ca2-a728-89b72b099a73 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:41.151442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ec55-e541-4ca2-a728-89b72b099a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:41.151739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ec55-e541-4ca2-a728-89b72b099a73 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:41.152393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4ec55-e541-4ca2-a728-89b72b099a73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:41.153749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4ec55-e541-4ca2-a728-89b72b099a73 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:41.161220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 635/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 07:45:41 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:43.237389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943fd7-5bbd-4749-81d5-af3776907354 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:43.237965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943fd7-5bbd-4749-81d5-af3776907354 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:43.237965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943fd7-5bbd-4749-81d5-af3776907354 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:43.239292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 667/1302] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:45:43.245288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fd6cc-f5bc-46e1-8d83-90c8aaacf4c8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:43.245696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fd6cc-f5bc-46e1-8d83-90c8aaacf4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:43.245921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fd6cc-f5bc-46e1-8d83-90c8aaacf4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:43.264304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-484fd6cc-f5bc-46e1-8d83-90c8aaacf4c8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:43.265540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-484fd6cc-f5bc-46e1-8d83-90c8aaacf4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:43.331418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 636/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:45:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:45:43.355412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943fd7-5bbd-4749-81d5-af3776907354 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:43.450330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:43.456896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943fd7-5bbd-4749-81d5-af3776907354 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:43.490523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:43.491089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:43.491369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:43.491671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:43.492289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:43.493660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:45:43.503388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:45:43.504467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Aug 01 07:45:43.505432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 668/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 07:45:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:45:43.510730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fd6cc-f5bc-46e1-8d83-90c8aaacf4c8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:43.587328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:43.610768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fd6cc-f5bc-46e1-8d83-90c8aaacf4c8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:43.643437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d101a-6647-45ed-a7f2-efa49c92fe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:43.644084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d101a-6647-45ed-a7f2-efa49c92fe0f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:43.644369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d101a-6647-45ed-a7f2-efa49c92fe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:43.644646 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d101a-6647-45ed-a7f2-efa49c92fe0f None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:43.645324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d101a-6647-45ed-a7f2-efa49c92fe0f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:43.645633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d101a-6647-45ed-a7f2-efa49c92fe0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:43.646996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d101a-6647-45ed-a7f2-efa49c92fe0f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:43.652737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 637/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 07:45:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:43.658786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:43.758654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e03b1-6653-4cc8-9e5f-49285589a29c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:43.793165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438468ee-0f17-430e-999d-947d55350004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:43.793781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438468ee-0f17-430e-999d-947d55350004 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:43.794063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438468ee-0f17-430e-999d-947d55350004 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:43.794338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438468ee-0f17-430e-999d-947d55350004 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:43.795046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438468ee-0f17-430e-999d-947d55350004 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:43.796451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438468ee-0f17-430e-999d-947d55350004 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:43.876339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 669/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:53.659225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=30 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:53.871794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:45.450647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691d3b1-a4dc-42f5-bbab-7b6355b47968 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:45.451698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691d3b1-a4dc-42f5-bbab-7b6355b47968 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:45.452189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691d3b1-a4dc-42f5-bbab-7b6355b47968 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:45.460470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf879de-fb04-4dae-93f8-cbeceb2763b0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:45.461442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf879de-fb04-4dae-93f8-cbeceb2763b0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:45.461928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf879de-fb04-4dae-93f8-cbeceb2763b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:45.466782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 638/1307] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:51:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:51:45.609409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bbf879de-fb04-4dae-93f8-cbeceb2763b0 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:51:45.611842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bbf879de-fb04-4dae-93f8-cbeceb2763b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:45.810317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 670/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:51:45.874433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df55a78-b16c-44bb-9135-3310b22e55bc None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:45.874989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df55a78-b16c-44bb-9135-3310b22e55bc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:45.875656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df55a78-b16c-44bb-9135-3310b22e55bc None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:45.878375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 639/1309] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 1 07:51:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:51:45.885640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc1218-27dd-45fb-8c47-c95eeb36e246 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:45.886534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc1218-27dd-45fb-8c47-c95eeb36e246 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:45.887732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc1218-27dd-45fb-8c47-c95eeb36e246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:46.101572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-74fc1218-27dd-45fb-8c47-c95eeb36e246 None None] MFA Rules not processed for user `ad3cf8a3e0a346fa867445d6b6300447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:51:46.102809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-74fc1218-27dd-45fb-8c47-c95eeb36e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:46.137904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:46.209836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 671/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 1 07:51:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:51:46.220591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df55a78-b16c-44bb-9135-3310b22e55bc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:51:46.224490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4df55a78-b16c-44bb-9135-3310b22e55bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:46.356321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df55a78-b16c-44bb-9135-3310b22e55bc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:51:46.436432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a603e6e-0148-4d8e-9557-c867913df652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:51:46.437346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a603e6e-0148-4d8e-9557-c867913df652 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:46.437802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a603e6e-0148-4d8e-9557-c867913df652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:46.438226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a603e6e-0148-4d8e-9557-c867913df652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:46.440125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6a603e6e-0148-4d8e-9557-c867913df652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:46.498230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:46.557424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a603e6e-0148-4d8e-9557-c867913df652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:51:46.558890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a603e6e-0148-4d8e-9557-c867913df652 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:51:46.585000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 640/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:51:48.940353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a233a-a834-4563-a9c6-1d487d8d2d03 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:48.940884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a233a-a834-4563-a9c6-1d487d8d2d03 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:48.940884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a233a-a834-4563-a9c6-1d487d8d2d03 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:48.942269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 672/1312] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:51:48.947816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461428a5-70df-42c0-8b75-c8af699e87e4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:48.948221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461428a5-70df-42c0-8b75-c8af699e87e4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:48.948774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461428a5-70df-42c0-8b75-c8af699e87e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:49.042900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-461428a5-70df-42c0-8b75-c8af699e87e4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:51:49.044081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-461428a5-70df-42c0-8b75-c8af699e87e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:49.147949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 641/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:51:49.202066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b2076-f530-4a3c-a33b-1c0754ff9a00 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:49.202361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b2076-f530-4a3c-a33b-1c0754ff9a00 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:49.202642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b2076-f530-4a3c-a33b-1c0754ff9a00 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:49.204103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 673/1314] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 1 07:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:51:49.210115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcfc54-9f81-41cf-b47c-606546f916d8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:49.210409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcfc54-9f81-41cf-b47c-606546f916d8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:49.210688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcfc54-9f81-41cf-b47c-606546f916d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:49.222481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:49.290076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fddcfc54-9f81-41cf-b47c-606546f916d8 None None] MFA Rules not processed for user `ad3cf8a3e0a346fa867445d6b6300447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:51:49.291347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fddcfc54-9f81-41cf-b47c-606546f916d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:49.370683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 642/1315] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 1 07:51:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:51:49.380649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b2076-f530-4a3c-a33b-1c0754ff9a00 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:51:49.382318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aa7b2076-f530-4a3c-a33b-1c0754ff9a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:49.485795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b2076-f530-4a3c-a33b-1c0754ff9a00 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:51:49.557012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:49.559586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53601be-7179-4ad3-877d-8efe910c97fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:51:49.560221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53601be-7179-4ad3-877d-8efe910c97fd service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:49.560508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53601be-7179-4ad3-877d-8efe910c97fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:49.560787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53601be-7179-4ad3-877d-8efe910c97fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:49.562466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a53601be-7179-4ad3-877d-8efe910c97fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:49.672249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53601be-7179-4ad3-877d-8efe910c97fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:51:49.674294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53601be-7179-4ad3-877d-8efe910c97fd service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:51:49.692313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 674/1316] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:51:52.384041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c8430-ab7c-4056-b943-ca562448436d None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:52.384584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c8430-ab7c-4056-b943-ca562448436d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:52.384584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c8430-ab7c-4056-b943-ca562448436d None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:52.385951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 643/1317] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:51:52.391621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395dfb0-3890-4372-9f02-d0ec6cc128cd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:52.392130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395dfb0-3890-4372-9f02-d0ec6cc128cd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:52.392412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395dfb0-3890-4372-9f02-d0ec6cc128cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:52.457612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5395dfb0-3890-4372-9f02-d0ec6cc128cd None None] MFA Rules not processed for user `5bbed68b31dc4a55a7039cbb96dc0c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:51:52.458825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5395dfb0-3890-4372-9f02-d0ec6cc128cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:52.510368 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:52.555506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 675/1318] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:51:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:51:52.597318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c8430-ab7c-4056-b943-ca562448436d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:51:52.598903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bc1c8430-ab7c-4056-b943-ca562448436d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:52.702907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c8430-ab7c-4056-b943-ca562448436d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:51:52.706572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:52.780353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d4262-47e3-4cc8-baea-364fde7ac2da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:51:52.780982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4262-47e3-4cc8-baea-364fde7ac2da service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:52.781263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4262-47e3-4cc8-baea-364fde7ac2da service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:52.781534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4262-47e3-4cc8-baea-364fde7ac2da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:52.783321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cf8d4262-47e3-4cc8-baea-364fde7ac2da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:52.893253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4262-47e3-4cc8-baea-364fde7ac2da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:51:52.894696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4262-47e3-4cc8-baea-364fde7ac2da service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:51:52.913172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 644/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:51:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:51:55.893150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76ea6a-2035-4597-875b-426077f6b70f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:55.893677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76ea6a-2035-4597-875b-426077f6b70f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:55.893677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76ea6a-2035-4597-875b-426077f6b70f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:55.895056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 676/1320] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:51:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:51:55.900068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e38dcc-5e3c-413e-acd7-6eb4d6973bd5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:55.900495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e38dcc-5e3c-413e-acd7-6eb4d6973bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:55.900630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e38dcc-5e3c-413e-acd7-6eb4d6973bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:55.942719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:55.966991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e2e38dcc-5e3c-413e-acd7-6eb4d6973bd5 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:51:55.968250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e2e38dcc-5e3c-413e-acd7-6eb4d6973bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:56.063864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 645/1321] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:51:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:51:56.102437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76ea6a-2035-4597-875b-426077f6b70f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:51:56.104031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8a76ea6a-2035-4597-875b-426077f6b70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:56.207836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76ea6a-2035-4597-875b-426077f6b70f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:51:56.279933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dec46d-2ba9-4963-ab06-73ce7030e8b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:51:56.280547 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec46d-2ba9-4963-ab06-73ce7030e8b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:56.280834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec46d-2ba9-4963-ab06-73ce7030e8b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:56.281117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec46d-2ba9-4963-ab06-73ce7030e8b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:56.283318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a8dec46d-2ba9-4963-ab06-73ce7030e8b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:56.290659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:56.392400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dec46d-2ba9-4963-ab06-73ce7030e8b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:51:56.393853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dec46d-2ba9-4963-ab06-73ce7030e8b0 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:51:56.412025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 677/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:51:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:51:59.455091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cb2ef-4db9-4458-a90f-f2abfed7d46a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:59.455638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cb2ef-4db9-4458-a90f-f2abfed7d46a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:59.455638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cb2ef-4db9-4458-a90f-f2abfed7d46a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:59.457191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 646/1323] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:51:59.462092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fbe45-f296-4209-93f0-301f326d2c66 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:59.462481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fbe45-f296-4209-93f0-301f326d2c66 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:59.462797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fbe45-f296-4209-93f0-301f326d2c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:59.531062 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4a6fbe45-f296-4209-93f0-301f326d2c66 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:51:59.532380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4a6fbe45-f296-4209-93f0-301f326d2c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:59.591250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:59.632188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 678/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:51:59.669984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cb2ef-4db9-4458-a90f-f2abfed7d46a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:51:59.671581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-480cb2ef-4db9-4458-a90f-f2abfed7d46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:59.790200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cb2ef-4db9-4458-a90f-f2abfed7d46a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:51:59.804079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:51:59.872189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99c2d3-b7a9-46c2-b351-c52f8be5e001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:51:59.872825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99c2d3-b7a9-46c2-b351-c52f8be5e001 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:51:59.873510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99c2d3-b7a9-46c2-b351-c52f8be5e001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:51:59.873510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99c2d3-b7a9-46c2-b351-c52f8be5e001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:51:59.875041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8e99c2d3-b7a9-46c2-b351-c52f8be5e001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:51:59.990857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99c2d3-b7a9-46c2-b351-c52f8be5e001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:51:59.992258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99c2d3-b7a9-46c2-b351-c52f8be5e001 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:00.011362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 647/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:02.061007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc16b-534e-4537-98b4-c064042a71aa None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:02.061545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc16b-534e-4537-98b4-c064042a71aa None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:02.061545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc16b-534e-4537-98b4-c064042a71aa None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:02.063692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 679/1326] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:02.068784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf047-b16e-4ab7-972f-0a128367cc79 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:02.069215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf047-b16e-4ab7-972f-0a128367cc79 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:02.069329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf047-b16e-4ab7-972f-0a128367cc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:02.118092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:02.135787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-cb3cf047-b16e-4ab7-972f-0a128367cc79 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:02.137036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cb3cf047-b16e-4ab7-972f-0a128367cc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:02.235226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 648/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:52:02.271965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc16b-534e-4537-98b4-c064042a71aa None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:02.273517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bc0dc16b-534e-4537-98b4-c064042a71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:02.380930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc16b-534e-4537-98b4-c064042a71aa None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:02.454116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c087283-d91f-4acd-b03e-8fc51f0e6f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:02.454739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c087283-d91f-4acd-b03e-8fc51f0e6f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:02.455025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c087283-d91f-4acd-b03e-8fc51f0e6f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:02.455296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c087283-d91f-4acd-b03e-8fc51f0e6f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:02.456978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4c087283-d91f-4acd-b03e-8fc51f0e6f25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:02.473377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:02.566939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c087283-d91f-4acd-b03e-8fc51f0e6f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:02.568416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c087283-d91f-4acd-b03e-8fc51f0e6f25 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:02.587764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 680/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:04.666292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d885d3-3041-456e-831f-60e2ff4cf145 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:04.666838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d885d3-3041-456e-831f-60e2ff4cf145 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:04.666838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d885d3-3041-456e-831f-60e2ff4cf145 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:04.668245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 649/1329] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:04.674272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01522d0b-e444-4be8-8957-740693ffdf44 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:04.674763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01522d0b-e444-4be8-8957-740693ffdf44 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:04.674876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01522d0b-e444-4be8-8957-740693ffdf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:04.748579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-01522d0b-e444-4be8-8957-740693ffdf44 None None] MFA Rules not processed for user `5bbed68b31dc4a55a7039cbb96dc0c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:04.749834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-01522d0b-e444-4be8-8957-740693ffdf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:04.823566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:04.856103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 681/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:04.894574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d885d3-3041-456e-831f-60e2ff4cf145 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:04.896175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-49d885d3-3041-456e-831f-60e2ff4cf145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:05.007317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d885d3-3041-456e-831f-60e2ff4cf145 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:05.029953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:05.085104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84143b55-2a16-497a-9227-f014a58cf75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:05.085721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84143b55-2a16-497a-9227-f014a58cf75e service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:05.086000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84143b55-2a16-497a-9227-f014a58cf75e service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:05.086266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84143b55-2a16-497a-9227-f014a58cf75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:05.087911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-84143b55-2a16-497a-9227-f014a58cf75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:05.196268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84143b55-2a16-497a-9227-f014a58cf75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:05.197628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84143b55-2a16-497a-9227-f014a58cf75e service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:05.216148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 650/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:07.793428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91801-216d-41d5-8941-2dbe04d4c756 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:07.793932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91801-216d-41d5-8941-2dbe04d4c756 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:07.794032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91801-216d-41d5-8941-2dbe04d4c756 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:07.795347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 682/1332] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:07.800593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee049e03-6cab-41f0-8608-ce919eb18abe None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:07.801020 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee049e03-6cab-41f0-8608-ce919eb18abe None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:07.801174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee049e03-6cab-41f0-8608-ce919eb18abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:07.864917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:07.867505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ee049e03-6cab-41f0-8608-ce919eb18abe None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:07.868747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ee049e03-6cab-41f0-8608-ce919eb18abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:07.965598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 651/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:08.001986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e91801-216d-41d5-8941-2dbe04d4c756 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:08.003594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67e91801-216d-41d5-8941-2dbe04d4c756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:08.104488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e91801-216d-41d5-8941-2dbe04d4c756 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:08.176649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e214ce7-b1b0-4183-b23e-eaa97ecf5144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:08.177263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e214ce7-b1b0-4183-b23e-eaa97ecf5144 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:08.177552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e214ce7-b1b0-4183-b23e-eaa97ecf5144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:08.177825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e214ce7-b1b0-4183-b23e-eaa97ecf5144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:08.179441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1e214ce7-b1b0-4183-b23e-eaa97ecf5144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:08.206163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:08.290033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e214ce7-b1b0-4183-b23e-eaa97ecf5144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:08.291481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e214ce7-b1b0-4183-b23e-eaa97ecf5144 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:08.310133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 683/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:52:10.364354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeabe6-7387-4b74-b154-b5ec0c9b18b6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:10.364914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeabe6-7387-4b74-b154-b5ec0c9b18b6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:10.364914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aeabe6-7387-4b74-b154-b5ec0c9b18b6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:10.366280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 652/1335] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:10.371346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97d1ca-ff9b-4b4b-b7bb-58332e220b59 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:10.371825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97d1ca-ff9b-4b4b-b7bb-58332e220b59 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:10.371919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97d1ca-ff9b-4b4b-b7bb-58332e220b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:10.437154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7a97d1ca-ff9b-4b4b-b7bb-58332e220b59 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:10.438933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7a97d1ca-ff9b-4b4b-b7bb-58332e220b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:10.513518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:10.535988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 684/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:52:10.575868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeabe6-7387-4b74-b154-b5ec0c9b18b6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:10.577410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34aeabe6-7387-4b74-b154-b5ec0c9b18b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:10.680916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aeabe6-7387-4b74-b154-b5ec0c9b18b6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:10.708889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:10.754377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122e0e5-5082-412f-b4ee-2015e0f75757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:10.754998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122e0e5-5082-412f-b4ee-2015e0f75757 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:10.755289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122e0e5-5082-412f-b4ee-2015e0f75757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:10.755578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122e0e5-5082-412f-b4ee-2015e0f75757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:10.757229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e122e0e5-5082-412f-b4ee-2015e0f75757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:10.865384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122e0e5-5082-412f-b4ee-2015e0f75757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:10.866824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122e0e5-5082-412f-b4ee-2015e0f75757 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:10.884910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 653/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:13.473648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a71efa-a1d2-486e-b914-47f18137fa97 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:13.474202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a71efa-a1d2-486e-b914-47f18137fa97 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:13.474202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a71efa-a1d2-486e-b914-47f18137fa97 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:13.475557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 685/1338] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:13.480821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48c6b6-ec58-4d18-9393-1d94372e8cb1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:13.481254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48c6b6-ec58-4d18-9393-1d94372e8cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:13.481376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48c6b6-ec58-4d18-9393-1d94372e8cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:13.551237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7f48c6b6-ec58-4d18-9393-1d94372e8cb1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:13.552666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7f48c6b6-ec58-4d18-9393-1d94372e8cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:13.560911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:13.649244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 654/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:52:13.686280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a71efa-a1d2-486e-b914-47f18137fa97 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:13.687922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-33a71efa-a1d2-486e-b914-47f18137fa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:13.791838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a71efa-a1d2-486e-b914-47f18137fa97 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:13.865566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8ffb3-5067-4052-9fda-77d403131301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:13.866178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ffb3-5067-4052-9fda-77d403131301 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:13.866462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ffb3-5067-4052-9fda-77d403131301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:13.866736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ffb3-5067-4052-9fda-77d403131301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:13.868399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-45d8ffb3-5067-4052-9fda-77d403131301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:13.903381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:13.978195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8ffb3-5067-4052-9fda-77d403131301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:13.979675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8ffb3-5067-4052-9fda-77d403131301 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:13.998004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 686/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:18.149327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fdf28-b80f-4b37-8520-283e1da71568 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:18.149817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fdf28-b80f-4b37-8520-283e1da71568 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:18.149913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fdf28-b80f-4b37-8520-283e1da71568 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:18.151235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 655/1341] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:18.157248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe2580-519d-498b-ab03-61a7d62e06ee None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:18.157646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe2580-519d-498b-ab03-61a7d62e06ee None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:18.157806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe2580-519d-498b-ab03-61a7d62e06ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:18.225273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f9fe2580-519d-498b-ab03-61a7d62e06ee None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:18.226481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f9fe2580-519d-498b-ab03-61a7d62e06ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:18.313636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:18.324883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 687/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:18.364236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fdf28-b80f-4b37-8520-283e1da71568 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:18.365816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0c7fdf28-b80f-4b37-8520-283e1da71568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:18.471125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fdf28-b80f-4b37-8520-283e1da71568 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:18.507413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:18.544971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e0223-7094-49c5-9a62-ca4330196184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:18.545597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e0223-7094-49c5-9a62-ca4330196184 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:18.545910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e0223-7094-49c5-9a62-ca4330196184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:18.546213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e0223-7094-49c5-9a62-ca4330196184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:18.547914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-659e0223-7094-49c5-9a62-ca4330196184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:18.660446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e0223-7094-49c5-9a62-ca4330196184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:18.661894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e0223-7094-49c5-9a62-ca4330196184 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:18.680623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 656/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:21.609611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78facfc-6fcb-475b-8d24-001e9be2de0f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:21.610255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78facfc-6fcb-475b-8d24-001e9be2de0f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:21.610255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78facfc-6fcb-475b-8d24-001e9be2de0f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:21.611570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 688/1344] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:21.617143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f407fc-2c0f-437a-9476-b555b748f043 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:21.620772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f407fc-2c0f-437a-9476-b555b748f043 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:21.620772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f407fc-2c0f-437a-9476-b555b748f043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:21.686431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-50f407fc-2c0f-437a-9476-b555b748f043 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:21.688162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-50f407fc-2c0f-437a-9476-b555b748f043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:21.707485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:21.791783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 657/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:21.829326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78facfc-6fcb-475b-8d24-001e9be2de0f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:21.830963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c78facfc-6fcb-475b-8d24-001e9be2de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:21.935184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78facfc-6fcb-475b-8d24-001e9be2de0f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:22.009239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754955c-77b9-4bab-98de-59fa22a52763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:22.009899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754955c-77b9-4bab-98de-59fa22a52763 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:22.010182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754955c-77b9-4bab-98de-59fa22a52763 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:22.010455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754955c-77b9-4bab-98de-59fa22a52763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:22.012097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7754955c-77b9-4bab-98de-59fa22a52763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:22.055737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:22.123898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754955c-77b9-4bab-98de-59fa22a52763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:22.125406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754955c-77b9-4bab-98de-59fa22a52763 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:22.147244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 689/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:26.401079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17b9c-20fe-4e3f-b4b3-06a27215738a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:26.401627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17b9c-20fe-4e3f-b4b3-06a27215738a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:26.401627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17b9c-20fe-4e3f-b4b3-06a27215738a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:26.403035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 658/1347] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:52:26.408786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5f8a8-50a4-4b80-9b1d-e3b789a057a3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:26.409272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5f8a8-50a4-4b80-9b1d-e3b789a057a3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:26.409362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5f8a8-50a4-4b80-9b1d-e3b789a057a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:26.479942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-30f5f8a8-50a4-4b80-9b1d-e3b789a057a3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:26.481168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-30f5f8a8-50a4-4b80-9b1d-e3b789a057a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:26.584032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:26.586754 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 690/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:26.626077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17b9c-20fe-4e3f-b4b3-06a27215738a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:26.627676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cce17b9c-20fe-4e3f-b4b3-06a27215738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:26.741595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17b9c-20fe-4e3f-b4b3-06a27215738a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:26.791510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:26.818727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedf491-2521-43d2-8e92-800c4599863c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:26.819400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedf491-2521-43d2-8e92-800c4599863c service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:26.819719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedf491-2521-43d2-8e92-800c4599863c service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:26.819999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedf491-2521-43d2-8e92-800c4599863c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:26.821676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0aedf491-2521-43d2-8e92-800c4599863c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:26.938735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedf491-2521-43d2-8e92-800c4599863c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:26.940210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedf491-2521-43d2-8e92-800c4599863c service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:26.959348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 659/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:31.931378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5df8e-ac41-4ca0-9672-0e3e85825c21 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:31.935965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5df8e-ac41-4ca0-9672-0e3e85825c21 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:31.935965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5df8e-ac41-4ca0-9672-0e3e85825c21 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:31.935965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 691/1350] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 07:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:31.939338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f3e87-a1e6-49ba-a3dc-2f7b4b77af12 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:31.940395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f3e87-a1e6-49ba-a3dc-2f7b4b77af12 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:31.940395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f3e87-a1e6-49ba-a3dc-2f7b4b77af12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:32.067465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ce8f3e87-a1e6-49ba-a3dc-2f7b4b77af12 None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:32.068752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ce8f3e87-a1e6-49ba-a3dc-2f7b4b77af12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:32.090216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:32.193374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 660/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 07:52:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:36.336488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7897f3-7cd8-4522-9eb4-d106b7d45f67 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:36.337058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7897f3-7cd8-4522-9eb4-d106b7d45f67 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:36.337058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7897f3-7cd8-4522-9eb4-d106b7d45f67 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:36.338557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 692/1352] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 07:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:36.344828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a9634-d5ed-415e-98c6-d39880c33354 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:36.345998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a9634-d5ed-415e-98c6-d39880c33354 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:36.346480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a9634-d5ed-415e-98c6-d39880c33354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:36.430436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-063a9634-d5ed-415e-98c6-d39880c33354 None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:36.431684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-063a9634-d5ed-415e-98c6-d39880c33354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:36.505091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:36.541863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 661/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 07:52:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:37.930676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997dcaf-8573-4d27-999a-39696a23b077 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:37.931212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997dcaf-8573-4d27-999a-39696a23b077 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:37.931212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997dcaf-8573-4d27-999a-39696a23b077 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:37.932828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 693/1354] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 07:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:37.938139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f353a-20d6-4a09-9231-d814d715748c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:37.938584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f353a-20d6-4a09-9231-d814d715748c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:37.938827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f353a-20d6-4a09-9231-d814d715748c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:38.020609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1b4f353a-20d6-4a09-9231-d814d715748c None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:38.021727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1b4f353a-20d6-4a09-9231-d814d715748c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:38.131044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 662/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 07:52:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:41.558972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa188f-f10e-4c5a-b28a-06a6da9a2523 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:41.559872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa188f-f10e-4c5a-b28a-06a6da9a2523 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:41.560333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa188f-f10e-4c5a-b28a-06a6da9a2523 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:41.561954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 694/1356] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:41.567415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db045-43e7-488e-8608-8ad8b4ba5187 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:41.567913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db045-43e7-488e-8608-8ad8b4ba5187 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:41.568029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db045-43e7-488e-8608-8ad8b4ba5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:41.616185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:41.651256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-669db045-43e7-488e-8608-8ad8b4ba5187 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:41.652710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-669db045-43e7-488e-8608-8ad8b4ba5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:41.767844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 663/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:41.821138 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc33b3-e657-4d2e-aa8f-62fd6f079df9 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:41.821990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc33b3-e657-4d2e-aa8f-62fd6f079df9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:41.822529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc33b3-e657-4d2e-aa8f-62fd6f079df9 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:41.824133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 695/1358] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 07:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:41.833263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96837-39d7-45a2-b5e9-d803b7b0b664 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:41.836951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96837-39d7-45a2-b5e9-d803b7b0b664 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:41.836951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96837-39d7-45a2-b5e9-d803b7b0b664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:41.925524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2da96837-39d7-45a2-b5e9-d803b7b0b664 None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:41.926719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2da96837-39d7-45a2-b5e9-d803b7b0b664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:41.981333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:42.051912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 664/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 07:52:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:42.059366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc33b3-e657-4d2e-aa8f-62fd6f079df9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:42.061481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7afc33b3-e657-4d2e-aa8f-62fd6f079df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:42.245755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc33b3-e657-4d2e-aa8f-62fd6f079df9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:42.325447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:42.369048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9974a-d596-47b8-b10b-3296e9d9e9c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:42.370047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9974a-d596-47b8-b10b-3296e9d9e9c5 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:42.370548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9974a-d596-47b8-b10b-3296e9d9e9c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:42.370982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9974a-d596-47b8-b10b-3296e9d9e9c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:42.374238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-26c9974a-d596-47b8-b10b-3296e9d9e9c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:42.478224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9974a-d596-47b8-b10b-3296e9d9e9c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:42.480281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9974a-d596-47b8-b10b-3296e9d9e9c5 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:42.501521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 696/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:44.731738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b54e76-0b7d-4a8d-968a-a68ff20b5519 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:44.732164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b54e76-0b7d-4a8d-968a-a68ff20b5519 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:44.733037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b54e76-0b7d-4a8d-968a-a68ff20b5519 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:44.734448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 665/1361] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:44.739922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf767ecb-798e-44d1-915c-e2cd6521c0bf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:44.740392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf767ecb-798e-44d1-915c-e2cd6521c0bf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:44.741087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf767ecb-798e-44d1-915c-e2cd6521c0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:44.823454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bf767ecb-798e-44d1-915c-e2cd6521c0bf None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:44.824818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bf767ecb-798e-44d1-915c-e2cd6521c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:44.871213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:44.940766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 697/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:44.985575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b54e76-0b7d-4a8d-968a-a68ff20b5519 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:44.988396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-15b54e76-0b7d-4a8d-968a-a68ff20b5519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:45.163366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b54e76-0b7d-4a8d-968a-a68ff20b5519 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:45.174658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:45.308409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a22b94-5a24-4f6d-b4dc-0c6d42269756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:45.308409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a22b94-5a24-4f6d-b4dc-0c6d42269756 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:45.309142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a22b94-5a24-4f6d-b4dc-0c6d42269756 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:45.309142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a22b94-5a24-4f6d-b4dc-0c6d42269756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:45.310846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a6a22b94-5a24-4f6d-b4dc-0c6d42269756 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:45.422413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a22b94-5a24-4f6d-b4dc-0c6d42269756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:45.424091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a22b94-5a24-4f6d-b4dc-0c6d42269756 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:45.444409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 666/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:47.907872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f042f-c0ce-4505-b81c-f318bfd12c7d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:47.908787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f042f-c0ce-4505-b81c-f318bfd12c7d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:47.909072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f042f-c0ce-4505-b81c-f318bfd12c7d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:47.910656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 698/1364] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:47.916098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a5d73-43c1-4d3c-b67c-65343fa417bf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:47.916899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a5d73-43c1-4d3c-b67c-65343fa417bf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:47.917376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a5d73-43c1-4d3c-b67c-65343fa417bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:47.934855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:47.998905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-647a5d73-43c1-4d3c-b67c-65343fa417bf None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:48.000346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-647a5d73-43c1-4d3c-b67c-65343fa417bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:48.110167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 667/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:48.157170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787e0d-366a-4330-a179-45af18e723a2 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:48.157661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787e0d-366a-4330-a179-45af18e723a2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:48.158007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787e0d-366a-4330-a179-45af18e723a2 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:48.159878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 699/1366] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 07:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:48.165215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c895a0e-aac3-4134-b0be-aa624519efd5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:48.166243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c895a0e-aac3-4134-b0be-aa624519efd5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:48.166243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c895a0e-aac3-4134-b0be-aa624519efd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:48.247154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6c895a0e-aac3-4134-b0be-aa624519efd5 None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:48.248969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6c895a0e-aac3-4134-b0be-aa624519efd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:48.285548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:48.387136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 668/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 07:52:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:48.395710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787e0d-366a-4330-a179-45af18e723a2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:48.397839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-86787e0d-366a-4330-a179-45af18e723a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:48.561174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787e0d-366a-4330-a179-45af18e723a2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:48.583605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:48.687798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df507186-9901-4ac2-8191-c2cab2f85678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:48.688390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507186-9901-4ac2-8191-c2cab2f85678 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:48.688672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507186-9901-4ac2-8191-c2cab2f85678 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:48.688950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507186-9901-4ac2-8191-c2cab2f85678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:48.690583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-df507186-9901-4ac2-8191-c2cab2f85678 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:48.792105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df507186-9901-4ac2-8191-c2cab2f85678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:48.793719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df507186-9901-4ac2-8191-c2cab2f85678 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:48.813132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 700/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:49.856671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6277c46-d8cc-4285-a2cb-ed2754e55069 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:49.857783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6277c46-d8cc-4285-a2cb-ed2754e55069 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:49.857783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6277c46-d8cc-4285-a2cb-ed2754e55069 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:49.859317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 669/1369] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:49.864330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b92b3-b07f-42c4-9d6e-d078e8769eb4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:49.864760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b92b3-b07f-42c4-9d6e-d078e8769eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:49.864974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b92b3-b07f-42c4-9d6e-d078e8769eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:49.898021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:49.934343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b48b92b3-b07f-42c4-9d6e-d078e8769eb4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:49.935662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b48b92b3-b07f-42c4-9d6e-d078e8769eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:50.040607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 701/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:52:50.385491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b081-28c5-4457-8754-aed98cb44bb6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:50.385491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b081-28c5-4457-8754-aed98cb44bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:50.385491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b081-28c5-4457-8754-aed98cb44bb6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:50.388260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 670/1371] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 1 07:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:50.391113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6567-c624-4b5d-8920-8f6b3a6e842f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:50.391406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6567-c624-4b5d-8920-8f6b3a6e842f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:50.391734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6567-c624-4b5d-8920-8f6b3a6e842f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:50.475508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-280a6567-c624-4b5d-8920-8f6b3a6e842f None None] MFA Rules not processed for user `317df4b0baa54808af9b7918ff5c3515`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:50.476910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-280a6567-c624-4b5d-8920-8f6b3a6e842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:50.525848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:50.557783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 702/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 1 07:52:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:50.565537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169b081-28c5-4457-8754-aed98cb44bb6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:50.567157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f169b081-28c5-4457-8754-aed98cb44bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:50.683819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169b081-28c5-4457-8754-aed98cb44bb6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:50.728854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:50.773849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86749d0-b92f-403d-a6d7-d7d015c03e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:50.774597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86749d0-b92f-403d-a6d7-d7d015c03e82 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:50.774881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86749d0-b92f-403d-a6d7-d7d015c03e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:50.775158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86749d0-b92f-403d-a6d7-d7d015c03e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:50.776899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c86749d0-b92f-403d-a6d7-d7d015c03e82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:50.897737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86749d0-b92f-403d-a6d7-d7d015c03e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:50.899199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86749d0-b92f-403d-a6d7-d7d015c03e82 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:50.921017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 671/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:51.057455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98ca2a-9f51-485d-bfa1-387698eea867 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:51.057969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98ca2a-9f51-485d-bfa1-387698eea867 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:51.058409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98ca2a-9f51-485d-bfa1-387698eea867 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:51.069817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 703/1374] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:51.070386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc972-9cfb-4b98-bc29-a653527051f7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:51.070386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc972-9cfb-4b98-bc29-a653527051f7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:51.070386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc972-9cfb-4b98-bc29-a653527051f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:51.137135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9a8fc972-9cfb-4b98-bc29-a653527051f7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:51.138929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9a8fc972-9cfb-4b98-bc29-a653527051f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:51.162790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:51.248536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 672/1375] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:52:51.293327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98ca2a-9f51-485d-bfa1-387698eea867 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:51.294980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4a98ca2a-9f51-485d-bfa1-387698eea867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:51.459455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98ca2a-9f51-485d-bfa1-387698eea867 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:51.512152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:51.575870 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8811313-e5f1-4c21-a9b9-27aea63158ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:51.576508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8811313-e5f1-4c21-a9b9-27aea63158ea service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:51.576803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8811313-e5f1-4c21-a9b9-27aea63158ea service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:51.577188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8811313-e5f1-4c21-a9b9-27aea63158ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:51.578860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c8811313-e5f1-4c21-a9b9-27aea63158ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:51.670693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8811313-e5f1-4c21-a9b9-27aea63158ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:51.672692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8811313-e5f1-4c21-a9b9-27aea63158ea service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:51.694582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 704/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:53.859015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3746e33-8b65-4c74-951d-40446c3f0ef5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:53.859569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3746e33-8b65-4c74-951d-40446c3f0ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:53.859569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3746e33-8b65-4c74-951d-40446c3f0ef5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:53.860946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 673/1377] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:53.866280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1d83e-0f5a-4c15-801e-751acf77b1dc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:53.866741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1d83e-0f5a-4c15-801e-751acf77b1dc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:53.866835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1d83e-0f5a-4c15-801e-751acf77b1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:53.936486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-dfc1d83e-0f5a-4c15-801e-751acf77b1dc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:53.937701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dfc1d83e-0f5a-4c15-801e-751acf77b1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:53.945578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:54.046511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 705/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:54.101458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3746e33-8b65-4c74-951d-40446c3f0ef5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:54.102951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f3746e33-8b65-4c74-951d-40446c3f0ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:54.251665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3746e33-8b65-4c74-951d-40446c3f0ef5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:54.256036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:54.381862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a936f-e9d8-473c-ac27-5940fc6183c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:54.382523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a936f-e9d8-473c-ac27-5940fc6183c8 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:54.382805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a936f-e9d8-473c-ac27-5940fc6183c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:54.383080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a936f-e9d8-473c-ac27-5940fc6183c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:54.384801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4f1a936f-e9d8-473c-ac27-5940fc6183c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:54.504170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a936f-e9d8-473c-ac27-5940fc6183c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:54.505843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a936f-e9d8-473c-ac27-5940fc6183c8 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:54.535633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 674/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:56.788076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1c24-9121-49c7-8afc-d84e95da3b22 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:56.788577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1c24-9121-49c7-8afc-d84e95da3b22 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:56.788716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1c24-9121-49c7-8afc-d84e95da3b22 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:56.791862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 706/1380] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:56.795708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39111-bc87-4146-afed-9ad47674a0b4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:56.796352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39111-bc87-4146-afed-9ad47674a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:56.796352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39111-bc87-4146-afed-9ad47674a0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:56.807800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:56.867851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6cc39111-bc87-4146-afed-9ad47674a0b4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:56.869063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6cc39111-bc87-4146-afed-9ad47674a0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:56.977750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 675/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:57.022370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d1c24-9121-49c7-8afc-d84e95da3b22 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:57.024124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9d7d1c24-9121-49c7-8afc-d84e95da3b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:57.170393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:57.181429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d1c24-9121-49c7-8afc-d84e95da3b22 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:52:57.310337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d2bb2-b398-4102-8935-87ea885042ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:52:57.311002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d2bb2-b398-4102-8935-87ea885042ea service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:57.311360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d2bb2-b398-4102-8935-87ea885042ea service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:57.311676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d2bb2-b398-4102-8935-87ea885042ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:57.313443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f04d2bb2-b398-4102-8935-87ea885042ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:57.431173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d2bb2-b398-4102-8935-87ea885042ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:52:57.432848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d2bb2-b398-4102-8935-87ea885042ea service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:52:57.444207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:52:57.454240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 707/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:52:59.680770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2977dd6-57c3-46e6-8c57-56776fdfd7f2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:59.681281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2977dd6-57c3-46e6-8c57-56776fdfd7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:59.681281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2977dd6-57c3-46e6-8c57-56776fdfd7f2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:59.682709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 676/1383] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:52:59.689716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1e435-f02f-4ebc-9717-0c6e2539edeb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:52:59.690140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1e435-f02f-4ebc-9717-0c6e2539edeb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:52:59.690309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1e435-f02f-4ebc-9717-0c6e2539edeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:52:59.764429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e0a1e435-f02f-4ebc-9717-0c6e2539edeb None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:52:59.765727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e0a1e435-f02f-4ebc-9717-0c6e2539edeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:52:59.875566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 708/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:52:59.924764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2977dd6-57c3-46e6-8c57-56776fdfd7f2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:52:59.926302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d2977dd6-57c3-46e6-8c57-56776fdfd7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:00.017035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:00.079053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2977dd6-57c3-46e6-8c57-56776fdfd7f2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:00.199371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84522da3-031b-407c-854e-148490228f9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:00.200117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522da3-031b-407c-854e-148490228f9e service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:00.200525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522da3-031b-407c-854e-148490228f9e service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:00.201087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522da3-031b-407c-854e-148490228f9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:00.202673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-84522da3-031b-407c-854e-148490228f9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:00.256867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:00.300738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84522da3-031b-407c-854e-148490228f9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:00.302334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84522da3-031b-407c-854e-148490228f9e service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:00.321882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 677/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:02.825326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7bb2e-fb68-447f-b75b-916330c2abbc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:02.825940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7bb2e-fb68-447f-b75b-916330c2abbc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:02.825940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7bb2e-fb68-447f-b75b-916330c2abbc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:02.827678 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 709/1386] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:02.833046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604c3ab-9d03-48a7-8cf9-755178358a98 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:02.833437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604c3ab-9d03-48a7-8cf9-755178358a98 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:02.833599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604c3ab-9d03-48a7-8cf9-755178358a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:02.900365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1604c3ab-9d03-48a7-8cf9-755178358a98 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:02.901587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1604c3ab-9d03-48a7-8cf9-755178358a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:03.000649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 678/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:03.057464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7bb2e-fb68-447f-b75b-916330c2abbc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:03.059012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2ed7bb2e-fb68-447f-b75b-916330c2abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:03.137422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:03.210559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7bb2e-fb68-447f-b75b-916330c2abbc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:03.333031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd300a5c-b3df-4327-b1cc-68a6ab0a1936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:03.333799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd300a5c-b3df-4327-b1cc-68a6ab0a1936 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:03.334154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd300a5c-b3df-4327-b1cc-68a6ab0a1936 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:03.334503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd300a5c-b3df-4327-b1cc-68a6ab0a1936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:03.336807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bd300a5c-b3df-4327-b1cc-68a6ab0a1936 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:03.377639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:03.436515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd300a5c-b3df-4327-b1cc-68a6ab0a1936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:03.438123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd300a5c-b3df-4327-b1cc-68a6ab0a1936 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:03.460705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 710/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:05.741525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23bbb-31b9-4c2d-89be-d1f834a16d20 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:05.742098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23bbb-31b9-4c2d-89be-d1f834a16d20 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:05.742098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23bbb-31b9-4c2d-89be-d1f834a16d20 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:05.743328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 679/1389] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:05.748824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281cd9a-9ff1-4031-839e-90dc9c626484 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:05.749260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281cd9a-9ff1-4031-839e-90dc9c626484 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:05.749366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281cd9a-9ff1-4031-839e-90dc9c626484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:05.817922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3281cd9a-9ff1-4031-839e-90dc9c626484 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:05.819118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3281cd9a-9ff1-4031-839e-90dc9c626484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:05.918216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 711/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:05.962571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d23bbb-31b9-4c2d-89be-d1f834a16d20 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:05.964133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-97d23bbb-31b9-4c2d-89be-d1f834a16d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:05.996292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:06.238513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4806ce2-ebe4-4180-b5be-6754e7f5ad74 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:06.238806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4806ce2-ebe4-4180-b5be-6754e7f5ad74 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:06.239089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4806ce2-ebe4-4180-b5be-6754e7f5ad74 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:06.240550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 712/1391] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:53:06.245492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc5917-c84d-48b2-bf7b-2a1db5c7c0ce None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:06.245785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc5917-c84d-48b2-bf7b-2a1db5c7c0ce None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:06.246064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc5917-c84d-48b2-bf7b-2a1db5c7c0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:06.248716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d23bbb-31b9-4c2d-89be-d1f834a16d20 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:06.263715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:06.317012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8fbc5917-c84d-48b2-bf7b-2a1db5c7c0ce None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:06.318227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8fbc5917-c84d-48b2-bf7b-2a1db5c7c0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:06.354804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:06.366197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:06.366815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:06.367124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:06.367403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:06.369069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:06.419235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 713/1392] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:53:06.463075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:06.464914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:06.475611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec78de-b65e-4942-a7c2-a157c72b20ee None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:06.475907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec78de-b65e-4942-a7c2-a157c72b20ee None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:06.476186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec78de-b65e-4942-a7c2-a157c72b20ee None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:06.477686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 714/1393] 10.222.0.48 () {54 vars in 966 bytes} [Mon Aug 1 07:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:53:06.482820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb6b8-e43b-4c0f-bde9-bc447a3f882a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:06.483106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb6b8-e43b-4c0f-bde9-bc447a3f882a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:06.483382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb6b8-e43b-4c0f-bde9-bc447a3f882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:06.484185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 680/1394] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:06.560233 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-78beb6b8-e43b-4c0f-bde9-bc447a3f882a None None] MFA Rules not processed for user `2f2160adab864eb9970b7767eeeb4789`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:06.561451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-78beb6b8-e43b-4c0f-bde9-bc447a3f882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:06.601407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:06.644483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 715/1395] 10.222.0.48 () {58 vars in 1063 bytes} [Mon Aug 1 07:53:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:53:06.652318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:06.663436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:06.740951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb392db-e4df-4d66-bbec-a6aeec1193ef service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:06.787639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:06.822898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab9e1d-a115-43f7-bc68-09a47193ad30 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:06.823444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab9e1d-a115-43f7-bc68-09a47193ad30 service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:06.823758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab9e1d-a115-43f7-bc68-09a47193ad30 service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:06.824040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab9e1d-a115-43f7-bc68-09a47193ad30 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:06.826212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-15ab9e1d-a115-43f7-bc68-09a47193ad30 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:06.940924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab9e1d-a115-43f7-bc68-09a47193ad30 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:06.942693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab9e1d-a115-43f7-bc68-09a47193ad30 service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:06.962089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 681/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:07.818634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edbde3-3523-4dde-b21d-80704369de51 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:07.819127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edbde3-3523-4dde-b21d-80704369de51 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:07.819250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edbde3-3523-4dde-b21d-80704369de51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:07.888641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-68edbde3-3523-4dde-b21d-80704369de51 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:07.889895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-68edbde3-3523-4dde-b21d-80704369de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:07.980301 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 716/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 07:53:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:08.026900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a9237-dd9f-4fcf-8185-cc7f949c9931 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:08.027413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a9237-dd9f-4fcf-8185-cc7f949c9931 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:08.027521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a9237-dd9f-4fcf-8185-cc7f949c9931 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:08.028870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 682/1398] 10.222.0.48 () {54 vars in 943 bytes} [Mon Aug 1 07:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:08.035338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aab011-9753-4af0-9cbe-1a2c761edeb1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:08.035656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aab011-9753-4af0-9cbe-1a2c761edeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:08.035928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aab011-9753-4af0-9cbe-1a2c761edeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:08.097497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:08.116133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c1aab011-9753-4af0-9cbe-1a2c761edeb1 None None] MFA Rules not processed for user `262558e0503a4f57a619c75acfdae331`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:08.117355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c1aab011-9753-4af0-9cbe-1a2c761edeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:08.225528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 717/1399] 10.222.0.48 () {58 vars in 1040 bytes} [Mon Aug 1 07:53:08 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:08.235537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a9237-dd9f-4fcf-8185-cc7f949c9931 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:08.237083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f25a9237-dd9f-4fcf-8185-cc7f949c9931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:08.314440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:08.375994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a9237-dd9f-4fcf-8185-cc7f949c9931 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:08.480973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b26cd7-e662-49b5-88cd-189f3d5bf0bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:08.482129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26cd7-e662-49b5-88cd-189f3d5bf0bd service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:08.482408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26cd7-e662-49b5-88cd-189f3d5bf0bd service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:08.482672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26cd7-e662-49b5-88cd-189f3d5bf0bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:08.484358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-76b26cd7-e662-49b5-88cd-189f3d5bf0bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:08.542838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:08.567193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b26cd7-e662-49b5-88cd-189f3d5bf0bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:08.568753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b26cd7-e662-49b5-88cd-189f3d5bf0bd service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:08.577921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 683/1400] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 07:53:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:53:08.643600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399156d0-dcf2-4054-beff-cdb047bc0b40 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:08.643898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399156d0-dcf2-4054-beff-cdb047bc0b40 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:08.644181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399156d0-dcf2-4054-beff-cdb047bc0b40 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:08.645614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 718/1401] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:08.650876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c894f-feba-406c-bd88-f56f7c3020b5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:08.651164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c894f-feba-406c-bd88-f56f7c3020b5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:08.654007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c894f-feba-406c-bd88-f56f7c3020b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:08.718219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-600c894f-feba-406c-bd88-f56f7c3020b5 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:08.719438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-600c894f-feba-406c-bd88-f56f7c3020b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:08.818032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 684/1402] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:08.862275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399156d0-dcf2-4054-beff-cdb047bc0b40 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:08.863868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-399156d0-dcf2-4054-beff-cdb047bc0b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:08.976307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:09.024892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399156d0-dcf2-4054-beff-cdb047bc0b40 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:09.157337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c28d-48e6-4adf-8995-b21de07a6df9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:09.158106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c28d-48e6-4adf-8995-b21de07a6df9 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:09.158467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c28d-48e6-4adf-8995-b21de07a6df9 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:09.158839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c28d-48e6-4adf-8995-b21de07a6df9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:09.160611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6032c28d-48e6-4adf-8995-b21de07a6df9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:09.238592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:09.261237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c28d-48e6-4adf-8995-b21de07a6df9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:09.262967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c28d-48e6-4adf-8995-b21de07a6df9 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:09.286763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 719/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:11.492289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e3770-d08d-473b-b7d1-7927fdbc4a9c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:11.492872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e3770-d08d-473b-b7d1-7927fdbc4a9c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:11.492872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e3770-d08d-473b-b7d1-7927fdbc4a9c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:11.494331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 685/1404] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:11.499912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efbdfa-28ef-4e21-bbd2-0d0f909b7857 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:11.500362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efbdfa-28ef-4e21-bbd2-0d0f909b7857 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:11.500468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efbdfa-28ef-4e21-bbd2-0d0f909b7857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:11.576846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-71efbdfa-28ef-4e21-bbd2-0d0f909b7857 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:11.578178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-71efbdfa-28ef-4e21-bbd2-0d0f909b7857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:11.687692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 720/1405] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:11.735492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e3770-d08d-473b-b7d1-7927fdbc4a9c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:11.737092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-005e3770-d08d-473b-b7d1-7927fdbc4a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:11.799612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:11.914119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e3770-d08d-473b-b7d1-7927fdbc4a9c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:12.046007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e9c35-9147-4ae7-a004-e29da00aa268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:12.046563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e9c35-9147-4ae7-a004-e29da00aa268 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:12.046851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e9c35-9147-4ae7-a004-e29da00aa268 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:12.047121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e9c35-9147-4ae7-a004-e29da00aa268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:12.048804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-013e9c35-9147-4ae7-a004-e29da00aa268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:12.061780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:12.152486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e9c35-9147-4ae7-a004-e29da00aa268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:12.154350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e9c35-9147-4ae7-a004-e29da00aa268 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:12.187625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 686/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:14.351251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915024-25d2-4855-8036-02c1a2e3f8ba None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:14.351857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915024-25d2-4855-8036-02c1a2e3f8ba None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:14.351857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915024-25d2-4855-8036-02c1a2e3f8ba None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:14.353353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 721/1407] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:14.358505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c40d6-3234-4f43-900c-a4f7c2e111aa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:14.359033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c40d6-3234-4f43-900c-a4f7c2e111aa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:14.359033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c40d6-3234-4f43-900c-a4f7c2e111aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:14.435344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1e9c40d6-3234-4f43-900c-a4f7c2e111aa None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:14.436936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e9c40d6-3234-4f43-900c-a4f7c2e111aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:14.542514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:14.558242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 687/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:53:14.606997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae915024-25d2-4855-8036-02c1a2e3f8ba None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:14.608707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ae915024-25d2-4855-8036-02c1a2e3f8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:14.672796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:14.784107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae915024-25d2-4855-8036-02c1a2e3f8ba None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:14.977566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db602c-cec5-417e-8a0e-5218e6a538d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:14.978676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db602c-cec5-417e-8a0e-5218e6a538d2 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:14.979196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db602c-cec5-417e-8a0e-5218e6a538d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:14.979724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db602c-cec5-417e-8a0e-5218e6a538d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:14.982407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-62db602c-cec5-417e-8a0e-5218e6a538d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:14.998909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:15.124181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db602c-cec5-417e-8a0e-5218e6a538d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:15.125869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db602c-cec5-417e-8a0e-5218e6a538d2 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:15.152311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 722/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:24.621329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:25.149703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=41 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:31.686098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff335ff-00dc-4e79-b956-61ec3f9e69fc None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:31.686638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff335ff-00dc-4e79-b956-61ec3f9e69fc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:31.686638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff335ff-00dc-4e79-b956-61ec3f9e69fc None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:31.688110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 688/1410] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 1 07:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:31.693036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c286f-a7ce-400d-940d-3ef03b7f66b8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:31.693478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c286f-a7ce-400d-940d-3ef03b7f66b8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:31.693585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c286f-a7ce-400d-940d-3ef03b7f66b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:31.771457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3d2c286f-a7ce-400d-940d-3ef03b7f66b8 None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:31.772700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3d2c286f-a7ce-400d-940d-3ef03b7f66b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:31.878856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 723/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 1 07:53:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:31.893436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe934d-ca70-4d55-9b92-441a980109f8 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:31.893725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe934d-ca70-4d55-9b92-441a980109f8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:31.894002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe934d-ca70-4d55-9b92-441a980109f8 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:31.895392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 689/1412] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 1 07:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:31.900080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf83b8-1458-4b7f-88f5-c72bf1183eaa None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:31.900377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf83b8-1458-4b7f-88f5-c72bf1183eaa None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:31.900660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf83b8-1458-4b7f-88f5-c72bf1183eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:31.972156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:31.975068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-eaaf83b8-1458-4b7f-88f5-c72bf1183eaa None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:31.976313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-eaaf83b8-1458-4b7f-88f5-c72bf1183eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:32.091420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 724/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 1 07:53:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:34.550416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83b1c-2255-435a-a4cf-5b0b17a28396 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:34.550975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83b1c-2255-435a-a4cf-5b0b17a28396 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:34.550975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83b1c-2255-435a-a4cf-5b0b17a28396 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:34.552255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 690/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:34.557670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11264614-6d8f-4f3b-956c-fc4888d33899 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:34.558102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11264614-6d8f-4f3b-956c-fc4888d33899 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:34.558239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11264614-6d8f-4f3b-956c-fc4888d33899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:34.624807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-11264614-6d8f-4f3b-956c-fc4888d33899 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:34.626032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-11264614-6d8f-4f3b-956c-fc4888d33899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:34.681751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:34.723855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 725/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:34.768435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83b1c-2255-435a-a4cf-5b0b17a28396 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:34.769788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-32e83b1c-2255-435a-a4cf-5b0b17a28396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:34.913434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83b1c-2255-435a-a4cf-5b0b17a28396 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:34.986986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:35.027815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32353ab8-f2dd-405e-be8c-67b17f5f187d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:35.028437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353ab8-f2dd-405e-be8c-67b17f5f187d service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:35.028719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353ab8-f2dd-405e-be8c-67b17f5f187d service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:35.028994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353ab8-f2dd-405e-be8c-67b17f5f187d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:35.030633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-32353ab8-f2dd-405e-be8c-67b17f5f187d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:35.119651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32353ab8-f2dd-405e-be8c-67b17f5f187d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:35.121241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32353ab8-f2dd-405e-be8c-67b17f5f187d service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:35.139415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 691/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:36.045779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed37d48-197e-4335-9674-dcc128bc48d8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:36.046318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed37d48-197e-4335-9674-dcc128bc48d8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:36.046318 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed37d48-197e-4335-9674-dcc128bc48d8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:36.047645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 726/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:36.052514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70b55e-e23c-4c3a-92f5-177bd90940f6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:36.052916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70b55e-e23c-4c3a-92f5-177bd90940f6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:36.053076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70b55e-e23c-4c3a-92f5-177bd90940f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:36.129951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-cb70b55e-e23c-4c3a-92f5-177bd90940f6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:36.131174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cb70b55e-e23c-4c3a-92f5-177bd90940f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:36.158086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:36.226756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 692/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:36.380366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed37d48-197e-4335-9674-dcc128bc48d8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:36.381914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6ed37d48-197e-4335-9674-dcc128bc48d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:36.526298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed37d48-197e-4335-9674-dcc128bc48d8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:36.566089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:36.640588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9356ae-c799-4f66-9f2e-bc76ca55c4db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:36.641131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9356ae-c799-4f66-9f2e-bc76ca55c4db service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:36.641418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9356ae-c799-4f66-9f2e-bc76ca55c4db service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:36.641694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9356ae-c799-4f66-9f2e-bc76ca55c4db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:36.643340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6f9356ae-c799-4f66-9f2e-bc76ca55c4db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:36.732824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9356ae-c799-4f66-9f2e-bc76ca55c4db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:36.734455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9356ae-c799-4f66-9f2e-bc76ca55c4db service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:36.752916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 727/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:37.627975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd66253-0d99-442a-89de-acf34685e1b1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:37.628535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd66253-0d99-442a-89de-acf34685e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:37.628535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd66253-0d99-442a-89de-acf34685e1b1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:37.629921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 693/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:37.634894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd15b0-0c28-426f-8a05-fe5043333b9b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:37.635344 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd15b0-0c28-426f-8a05-fe5043333b9b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:37.635443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd15b0-0c28-426f-8a05-fe5043333b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:37.691153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:37.713173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3cfd15b0-0c28-426f-8a05-fe5043333b9b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:37.714619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3cfd15b0-0c28-426f-8a05-fe5043333b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:37.936122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 728/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:38.090421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd66253-0d99-442a-89de-acf34685e1b1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:38.091928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4dd66253-0d99-442a-89de-acf34685e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:38.238198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd66253-0d99-442a-89de-acf34685e1b1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:38.251056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:38.352551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2758619-f8a4-449c-8f3a-54a9e2ca26bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:38.353086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758619-f8a4-449c-8f3a-54a9e2ca26bf service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:38.353365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758619-f8a4-449c-8f3a-54a9e2ca26bf service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:38.353635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758619-f8a4-449c-8f3a-54a9e2ca26bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:38.356256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e2758619-f8a4-449c-8f3a-54a9e2ca26bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:38.446379 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2758619-f8a4-449c-8f3a-54a9e2ca26bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:38.448000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2758619-f8a4-449c-8f3a-54a9e2ca26bf service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:38.466532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 694/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:39.328821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0bc74-fc74-4b76-b11d-438c03f58eef None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:39.329422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0bc74-fc74-4b76-b11d-438c03f58eef None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:39.329422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0bc74-fc74-4b76-b11d-438c03f58eef None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:39.330724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 729/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:39.335616 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f40272-add6-4fa9-b3c0-3e2270255148 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:39.336118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f40272-add6-4fa9-b3c0-3e2270255148 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:39.336216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f40272-add6-4fa9-b3c0-3e2270255148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:39.356148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:39.412089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-79f40272-add6-4fa9-b3c0-3e2270255148 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:39.413300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-79f40272-add6-4fa9-b3c0-3e2270255148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:39.509281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 695/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:39.660912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0bc74-fc74-4b76-b11d-438c03f58eef None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:39.662476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-64f0bc74-fc74-4b76-b11d-438c03f58eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:39.773823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:39.810740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0bc74-fc74-4b76-b11d-438c03f58eef None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:39.921903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf89af-3e40-4937-8db2-4c29f4808d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:39.922453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf89af-3e40-4937-8db2-4c29f4808d9a service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:39.922736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf89af-3e40-4937-8db2-4c29f4808d9a service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:39.923008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf89af-3e40-4937-8db2-4c29f4808d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:39.924676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-efcf89af-3e40-4937-8db2-4c29f4808d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:39.998962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:40.012203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf89af-3e40-4937-8db2-4c29f4808d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:40.013808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf89af-3e40-4937-8db2-4c29f4808d9a service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:40.031953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 730/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:40.893939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea4aa1-efa2-44de-a669-da9450962869 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:40.894477 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea4aa1-efa2-44de-a669-da9450962869 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:40.894477 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea4aa1-efa2-44de-a669-da9450962869 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:40.895906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 696/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:40.900853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d70b18-6a22-4478-9ff4-65f9bcd1bf69 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:40.901303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d70b18-6a22-4478-9ff4-65f9bcd1bf69 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:40.901417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d70b18-6a22-4478-9ff4-65f9bcd1bf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:40.977960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-06d70b18-6a22-4478-9ff4-65f9bcd1bf69 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:40.979175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-06d70b18-6a22-4478-9ff4-65f9bcd1bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:41.086084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 731/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:41.237459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea4aa1-efa2-44de-a669-da9450962869 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:41.238975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-23ea4aa1-efa2-44de-a669-da9450962869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:41.324617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:41.385651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea4aa1-efa2-44de-a669-da9450962869 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:41.497392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7a4bf-e9c0-4821-8440-f0039fcd9741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:41.498008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a4bf-e9c0-4821-8440-f0039fcd9741 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:41.498287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a4bf-e9c0-4821-8440-f0039fcd9741 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:41.498557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a4bf-e9c0-4821-8440-f0039fcd9741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:41.500199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8fb7a4bf-e9c0-4821-8440-f0039fcd9741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:41.547719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:41.587954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7a4bf-e9c0-4821-8440-f0039fcd9741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:41.590100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7a4bf-e9c0-4821-8440-f0039fcd9741 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:41.608961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 697/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:42.476461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8d067-b091-45e0-add5-0fc0abf70a6a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:42.477007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8d067-b091-45e0-add5-0fc0abf70a6a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:42.477007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8d067-b091-45e0-add5-0fc0abf70a6a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:42.478360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 732/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:42.484012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b33603-0797-4430-8c80-9bd433e605ac None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:42.484444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b33603-0797-4430-8c80-9bd433e605ac None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:42.484579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b33603-0797-4430-8c80-9bd433e605ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:42.560183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b8b33603-0797-4430-8c80-9bd433e605ac None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:42.561402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b8b33603-0797-4430-8c80-9bd433e605ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:42.657174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 698/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:42.811341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8d067-b091-45e0-add5-0fc0abf70a6a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:42.812894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-45a8d067-b091-45e0-add5-0fc0abf70a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:42.861165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:42.957186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8d067-b091-45e0-add5-0fc0abf70a6a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:43.068566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d88227-8d18-4cbb-bb00-130407be7560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:43.069112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d88227-8d18-4cbb-bb00-130407be7560 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:43.069663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d88227-8d18-4cbb-bb00-130407be7560 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:43.069939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d88227-8d18-4cbb-bb00-130407be7560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:43.071590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-21d88227-8d18-4cbb-bb00-130407be7560 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:43.078982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:43.159455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d88227-8d18-4cbb-bb00-130407be7560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:43.161107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d88227-8d18-4cbb-bb00-130407be7560 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:43.179010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 733/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:44.070912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e01a-0d51-4e26-b5b9-3bfff04466c3 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:44.071437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e01a-0d51-4e26-b5b9-3bfff04466c3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:44.071437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e01a-0d51-4e26-b5b9-3bfff04466c3 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:44.072813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 699/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:44.077859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559e6d6-1673-4aa1-8931-145a0100e2e5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:44.078276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559e6d6-1673-4aa1-8931-145a0100e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:44.078421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559e6d6-1673-4aa1-8931-145a0100e2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:44.153863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6559e6d6-1673-4aa1-8931-145a0100e2e5 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:44.155056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6559e6d6-1673-4aa1-8931-145a0100e2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:44.223232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:44.253389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 734/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:44.406136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e01a-0d51-4e26-b5b9-3bfff04466c3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:44.407747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2a98e01a-0d51-4e26-b5b9-3bfff04466c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:44.425794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:44.558240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e01a-0d51-4e26-b5b9-3bfff04466c3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:44.649095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:44.677382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c8ea8-2b7e-43c6-87ad-250c1b59f7e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:44.678009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8ea8-2b7e-43c6-87ad-250c1b59f7e7 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:44.678299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8ea8-2b7e-43c6-87ad-250c1b59f7e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:44.678575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8ea8-2b7e-43c6-87ad-250c1b59f7e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:44.680261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c11c8ea8-2b7e-43c6-87ad-250c1b59f7e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:44.771757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c8ea8-2b7e-43c6-87ad-250c1b59f7e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:44.773359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c8ea8-2b7e-43c6-87ad-250c1b59f7e7 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:44.791126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 700/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:45.666484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc5f1-797b-4cc5-8cd0-5e04d91303ef None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:45.667021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc5f1-797b-4cc5-8cd0-5e04d91303ef None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:45.667371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc5f1-797b-4cc5-8cd0-5e04d91303ef None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:45.668773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 735/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:45.673762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c149a-ca2b-4890-a57b-389676929d2d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:45.674220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c149a-ca2b-4890-a57b-389676929d2d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:45.674319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c149a-ca2b-4890-a57b-389676929d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:45.751437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e92c149a-ca2b-4890-a57b-389676929d2d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:45.752622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e92c149a-ca2b-4890-a57b-389676929d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:45.796397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:45.852262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 701/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:46.006495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccc5f1-797b-4cc5-8cd0-5e04d91303ef None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:46.007984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e7ccc5f1-797b-4cc5-8cd0-5e04d91303ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:46.154329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccc5f1-797b-4cc5-8cd0-5e04d91303ef None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:46.207916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:46.267049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e49af7-ef25-4b98-ae4c-a8d29f746b2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:46.267635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e49af7-ef25-4b98-ae4c-a8d29f746b2e service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:46.267921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e49af7-ef25-4b98-ae4c-a8d29f746b2e service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:46.268202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e49af7-ef25-4b98-ae4c-a8d29f746b2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:46.269830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-58e49af7-ef25-4b98-ae4c-a8d29f746b2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:46.358422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e49af7-ef25-4b98-ae4c-a8d29f746b2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:46.360049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e49af7-ef25-4b98-ae4c-a8d29f746b2e service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:46.378097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 736/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:47.264535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8215-8147-4c50-8281-99247e8a6177 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:47.265129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8215-8147-4c50-8281-99247e8a6177 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:47.265129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8215-8147-4c50-8281-99247e8a6177 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:47.266754 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 702/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:47.272062 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a963a1-b975-4e64-b266-3a5ad3ea1f6c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:47.272508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a963a1-b975-4e64-b266-3a5ad3ea1f6c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:47.272627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a963a1-b975-4e64-b266-3a5ad3ea1f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:47.356832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c7a963a1-b975-4e64-b266-3a5ad3ea1f6c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:47.358054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c7a963a1-b975-4e64-b266-3a5ad3ea1f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:47.360302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:47.460304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 737/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:47.631259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed8215-8147-4c50-8281-99247e8a6177 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:47.632844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-aeed8215-8147-4c50-8281-99247e8a6177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:47.789631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed8215-8147-4c50-8281-99247e8a6177 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:47.818677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:47.906959 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cab85-cf5c-42ba-a7e3-6c9ae3a3adf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:47.907571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cab85-cf5c-42ba-a7e3-6c9ae3a3adf1 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:47.907861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cab85-cf5c-42ba-a7e3-6c9ae3a3adf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:47.908138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cab85-cf5c-42ba-a7e3-6c9ae3a3adf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:47.909803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3e9cab85-cf5c-42ba-a7e3-6c9ae3a3adf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:48.000427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cab85-cf5c-42ba-a7e3-6c9ae3a3adf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:48.002085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cab85-cf5c-42ba-a7e3-6c9ae3a3adf1 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:48.020581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 703/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:49.938486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2c18-37d8-462f-a34d-8583b5672fa9 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:49.939094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2c18-37d8-462f-a34d-8583b5672fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:49.939094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2c18-37d8-462f-a34d-8583b5672fa9 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:49.940444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 738/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:49.945982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f29a-eb89-4821-aa91-e61df9493d6b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:49.946391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f29a-eb89-4821-aa91-e61df9493d6b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:49.946541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f29a-eb89-4821-aa91-e61df9493d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:49.988397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:50.014141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5fd5f29a-eb89-4821-aa91-e61df9493d6b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:50.015299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5fd5f29a-eb89-4821-aa91-e61df9493d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:50.112059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 704/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:50.150119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b2c18-37d8-462f-a34d-8583b5672fa9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:50.152373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ea5b2c18-37d8-462f-a34d-8583b5672fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:50.254001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b2c18-37d8-462f-a34d-8583b5672fa9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:50.285745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:50.327259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f8a41-8850-4f52-a31a-350448d70373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:50.327914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f8a41-8850-4f52-a31a-350448d70373 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:50.328251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f8a41-8850-4f52-a31a-350448d70373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:50.328530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f8a41-8850-4f52-a31a-350448d70373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:50.330149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a98f8a41-8850-4f52-a31a-350448d70373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:50.436529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f8a41-8850-4f52-a31a-350448d70373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:50.437966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f8a41-8850-4f52-a31a-350448d70373 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:50.456446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 739/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:52.703420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e880a-5d21-4af4-85d6-6c5c278a6538 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:52.704075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e880a-5d21-4af4-85d6-6c5c278a6538 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:52.704075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e880a-5d21-4af4-85d6-6c5c278a6538 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:52.705336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 705/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:52.710522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e62c-c4da-4e1b-9651-d067c8e22cba None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:52.710979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e62c-c4da-4e1b-9651-d067c8e22cba None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:52.711083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e62c-c4da-4e1b-9651-d067c8e22cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:52.779471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cef9e62c-c4da-4e1b-9651-d067c8e22cba None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:52.780735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cef9e62c-c4da-4e1b-9651-d067c8e22cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:52.793501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:52.878208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 740/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:52.917084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e880a-5d21-4af4-85d6-6c5c278a6538 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:52.918631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a65e880a-5d21-4af4-85d6-6c5c278a6538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:53.023723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e880a-5d21-4af4-85d6-6c5c278a6538 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:53.035792 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:53.097663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23cc95-fc7f-49c7-984a-18545d59f3af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:53.098280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23cc95-fc7f-49c7-984a-18545d59f3af service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:53.098565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23cc95-fc7f-49c7-984a-18545d59f3af service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:53.098839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23cc95-fc7f-49c7-984a-18545d59f3af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:53.100485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fb23cc95-fc7f-49c7-984a-18545d59f3af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:53.209656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23cc95-fc7f-49c7-984a-18545d59f3af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:53.211093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23cc95-fc7f-49c7-984a-18545d59f3af service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:53.229876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 706/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:55.723748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dd137-1dbc-46f8-a195-da1cdcd81970 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:55.724280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dd137-1dbc-46f8-a195-da1cdcd81970 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:55.724280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dd137-1dbc-46f8-a195-da1cdcd81970 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:55.725553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 741/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:55.731372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3176-b911-42f6-9993-7b4c874bc415 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:55.731795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3176-b911-42f6-9993-7b4c874bc415 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:55.731986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3176-b911-42f6-9993-7b4c874bc415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:55.781518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:55.799102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9fdc3176-b911-42f6-9993-7b4c874bc415 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:55.800367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9fdc3176-b911-42f6-9993-7b4c874bc415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:55.899418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 707/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:55.938857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491dd137-1dbc-46f8-a195-da1cdcd81970 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:55.940490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-491dd137-1dbc-46f8-a195-da1cdcd81970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:56.043720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491dd137-1dbc-46f8-a195-da1cdcd81970 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:56.083955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:56.118295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5abff7-cde0-47df-a4c9-fee04a67ca12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:56.118915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5abff7-cde0-47df-a4c9-fee04a67ca12 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:56.119996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5abff7-cde0-47df-a4c9-fee04a67ca12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:56.120295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5abff7-cde0-47df-a4c9-fee04a67ca12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:56.121965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4c5abff7-cde0-47df-a4c9-fee04a67ca12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:56.229902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5abff7-cde0-47df-a4c9-fee04a67ca12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:56.231343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5abff7-cde0-47df-a4c9-fee04a67ca12 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:56.250097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 742/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:53:58.374341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8879cf-fbcd-4118-8337-8ed6d8647806 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:58.374909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8879cf-fbcd-4118-8337-8ed6d8647806 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:58.374909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8879cf-fbcd-4118-8337-8ed6d8647806 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:58.376287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 708/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:53:58.381512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b480e-7a9e-4330-b8e9-795b85291b51 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:58.381969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b480e-7a9e-4330-b8e9-795b85291b51 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:58.382079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b480e-7a9e-4330-b8e9-795b85291b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:58.448309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-018b480e-7a9e-4330-b8e9-795b85291b51 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:53:58.449564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-018b480e-7a9e-4330-b8e9-795b85291b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:58.470306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:58.546485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 743/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:53:58.585026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8879cf-fbcd-4118-8337-8ed6d8647806 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:53:58.589431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e8879cf-fbcd-4118-8337-8ed6d8647806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:58.691868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8879cf-fbcd-4118-8337-8ed6d8647806 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:53:58.712511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:53:58.768125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ce883-45a5-4875-9128-ee7680afa0c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:53:58.768748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ce883-45a5-4875-9128-ee7680afa0c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:53:58.769028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ce883-45a5-4875-9128-ee7680afa0c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:53:58.769298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ce883-45a5-4875-9128-ee7680afa0c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:53:58.770911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-926ce883-45a5-4875-9128-ee7680afa0c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:53:58.878429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ce883-45a5-4875-9128-ee7680afa0c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:53:58.879886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ce883-45a5-4875-9128-ee7680afa0c2 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:53:58.897961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 709/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:00.956658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea51b-7b44-4a3d-ae5f-07f6d196ae7f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:00.957215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea51b-7b44-4a3d-ae5f-07f6d196ae7f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:00.957215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea51b-7b44-4a3d-ae5f-07f6d196ae7f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:00.958582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 744/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:00.964498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07851510-c9b2-4efe-aa4d-86058cdd7f6a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:00.964933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07851510-c9b2-4efe-aa4d-86058cdd7f6a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:00.965041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07851510-c9b2-4efe-aa4d-86058cdd7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:01.028929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:01.031501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-07851510-c9b2-4efe-aa4d-86058cdd7f6a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:01.032745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-07851510-c9b2-4efe-aa4d-86058cdd7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:01.130159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 710/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:01.169285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea51b-7b44-4a3d-ae5f-07f6d196ae7f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:01.170878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9f3ea51b-7b44-4a3d-ae5f-07f6d196ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:01.274519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea51b-7b44-4a3d-ae5f-07f6d196ae7f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:01.323188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:01.347591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34d098-f1a8-4ae8-9cc8-55bb291e3acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:01.348224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34d098-f1a8-4ae8-9cc8-55bb291e3acd service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:01.348499 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34d098-f1a8-4ae8-9cc8-55bb291e3acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:01.348777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34d098-f1a8-4ae8-9cc8-55bb291e3acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:01.351150 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1d34d098-f1a8-4ae8-9cc8-55bb291e3acd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:01.461047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34d098-f1a8-4ae8-9cc8-55bb291e3acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:01.463192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34d098-f1a8-4ae8-9cc8-55bb291e3acd service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:01.481977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 745/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:03.600690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c73fdf-af15-44c2-86b5-7e923b829f06 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:03.601105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c73fdf-af15-44c2-86b5-7e923b829f06 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:03.601224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c73fdf-af15-44c2-86b5-7e923b829f06 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:03.602580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 711/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:03.608456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394775ba-c4cd-437e-9920-68b1f8c02adc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:03.608854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394775ba-c4cd-437e-9920-68b1f8c02adc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:03.609016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394775ba-c4cd-437e-9920-68b1f8c02adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:03.677157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-394775ba-c4cd-437e-9920-68b1f8c02adc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:03.678369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-394775ba-c4cd-437e-9920-68b1f8c02adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:03.708480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:03.779460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 746/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:03.817470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c73fdf-af15-44c2-86b5-7e923b829f06 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:03.819285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b8c73fdf-af15-44c2-86b5-7e923b829f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:03.923899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c73fdf-af15-44c2-86b5-7e923b829f06 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:03.951781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:03.997952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7395a84-d94c-4a0d-8fa6-ad24643d40ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:03.998570 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395a84-d94c-4a0d-8fa6-ad24643d40ce service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:03.998854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395a84-d94c-4a0d-8fa6-ad24643d40ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:03.999129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395a84-d94c-4a0d-8fa6-ad24643d40ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:04.000790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e7395a84-d94c-4a0d-8fa6-ad24643d40ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:04.109107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395a84-d94c-4a0d-8fa6-ad24643d40ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:04.110462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395a84-d94c-4a0d-8fa6-ad24643d40ce service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:04.128697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 712/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:06.216216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee962ab-250a-4057-be7a-5cb0b84ae347 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:06.216788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee962ab-250a-4057-be7a-5cb0b84ae347 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:06.216788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee962ab-250a-4057-be7a-5cb0b84ae347 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:06.218146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 747/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:54:06.223338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcd77-575a-44d9-b2a7-02f03e7366ee None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:06.223807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcd77-575a-44d9-b2a7-02f03e7366ee None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:06.223917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcd77-575a-44d9-b2a7-02f03e7366ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:06.289397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4d1dcd77-575a-44d9-b2a7-02f03e7366ee None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:06.291260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4d1dcd77-575a-44d9-b2a7-02f03e7366ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:06.299601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:06.387203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 713/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:06.423983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee962ab-250a-4057-be7a-5cb0b84ae347 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:06.425584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8ee962ab-250a-4057-be7a-5cb0b84ae347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:06.530421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee962ab-250a-4057-be7a-5cb0b84ae347 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:06.586627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:06.606482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c511a1-b3ee-4b20-ab70-92bf3721ae14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:06.607108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c511a1-b3ee-4b20-ab70-92bf3721ae14 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:06.607396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c511a1-b3ee-4b20-ab70-92bf3721ae14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:06.607719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c511a1-b3ee-4b20-ab70-92bf3721ae14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:06.609362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-99c511a1-b3ee-4b20-ab70-92bf3721ae14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:06.718655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c511a1-b3ee-4b20-ab70-92bf3721ae14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:06.720136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c511a1-b3ee-4b20-ab70-92bf3721ae14 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:06.738834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 748/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:08.797010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81632-98c6-471b-be63-326213e8f412 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:08.797532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81632-98c6-471b-be63-326213e8f412 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:08.797532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81632-98c6-471b-be63-326213e8f412 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:08.798917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 714/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:08.804277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f5cc-bd6e-466e-8ad8-063e160ee88d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:08.804726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f5cc-bd6e-466e-8ad8-063e160ee88d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:08.804867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f5cc-bd6e-466e-8ad8-063e160ee88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:08.873029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e697f5cc-bd6e-466e-8ad8-063e160ee88d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:08.874259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e697f5cc-bd6e-466e-8ad8-063e160ee88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:08.912535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:08.971252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 749/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:09.008424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81632-98c6-471b-be63-326213e8f412 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:09.010622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b1b81632-98c6-471b-be63-326213e8f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:09.113511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81632-98c6-471b-be63-326213e8f412 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:09.151637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:09.188106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353a931-8218-409b-8d04-0ce6ffbe44ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:09.188728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353a931-8218-409b-8d04-0ce6ffbe44ba service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:09.189016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353a931-8218-409b-8d04-0ce6ffbe44ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:09.189293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353a931-8218-409b-8d04-0ce6ffbe44ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:09.190922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6353a931-8218-409b-8d04-0ce6ffbe44ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:09.298597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353a931-8218-409b-8d04-0ce6ffbe44ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:09.300074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353a931-8218-409b-8d04-0ce6ffbe44ba service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:09.320383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 715/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:11.338492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0e39-f3d3-491a-9efc-8a44b426b49d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:11.339064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0e39-f3d3-491a-9efc-8a44b426b49d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:11.339064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0e39-f3d3-491a-9efc-8a44b426b49d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:11.340457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 750/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:11.345663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b11b-8223-4bc8-b45b-c3f5251a2698 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:11.346075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b11b-8223-4bc8-b45b-c3f5251a2698 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:11.346218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b11b-8223-4bc8-b45b-c3f5251a2698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:11.413434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1301b11b-8223-4bc8-b45b-c3f5251a2698 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:11.414700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1301b11b-8223-4bc8-b45b-c3f5251a2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:11.433686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:11.512600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 716/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:11.549682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0e39-f3d3-491a-9efc-8a44b426b49d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:11.551511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c28a0e39-f3d3-491a-9efc-8a44b426b49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:11.662382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0e39-f3d3-491a-9efc-8a44b426b49d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:11.728440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:11.735093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b74d7-fcb4-46f5-b4b0-16e1cbbbc0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:11.735738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b74d7-fcb4-46f5-b4b0-16e1cbbbc0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:11.736072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b74d7-fcb4-46f5-b4b0-16e1cbbbc0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:11.736290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b74d7-fcb4-46f5-b4b0-16e1cbbbc0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:11.737933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-821b74d7-fcb4-46f5-b4b0-16e1cbbbc0d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:11.847327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b74d7-fcb4-46f5-b4b0-16e1cbbbc0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:11.848831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b74d7-fcb4-46f5-b4b0-16e1cbbbc0d3 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:11.867251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 751/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:13.891376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9898e62-1fd5-465f-ad77-54e1b0b82a58 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:13.891983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9898e62-1fd5-465f-ad77-54e1b0b82a58 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:13.891983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9898e62-1fd5-465f-ad77-54e1b0b82a58 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:13.893333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 717/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:13.898698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74fa1-882f-4a4c-b0e6-b4e7ffe3d140 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:13.899098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74fa1-882f-4a4c-b0e6-b4e7ffe3d140 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:13.899251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc74fa1-882f-4a4c-b0e6-b4e7ffe3d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:13.965179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0fc74fa1-882f-4a4c-b0e6-b4e7ffe3d140 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:13.966381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0fc74fa1-882f-4a4c-b0e6-b4e7ffe3d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:14.014286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:14.062757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 752/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:14.099068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9898e62-1fd5-465f-ad77-54e1b0b82a58 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:14.100650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a9898e62-1fd5-465f-ad77-54e1b0b82a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:14.206260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9898e62-1fd5-465f-ad77-54e1b0b82a58 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:14.254668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:14.282676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fa4f7-9fb8-4b6b-ab0e-61ebac68ba74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:14.283227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fa4f7-9fb8-4b6b-ab0e-61ebac68ba74 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:14.283510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fa4f7-9fb8-4b6b-ab0e-61ebac68ba74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:14.283817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fa4f7-9fb8-4b6b-ab0e-61ebac68ba74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:14.285445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-046fa4f7-9fb8-4b6b-ab0e-61ebac68ba74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:14.393720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fa4f7-9fb8-4b6b-ab0e-61ebac68ba74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:14.395153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fa4f7-9fb8-4b6b-ab0e-61ebac68ba74 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:14.413342 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 718/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:24.060998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=22 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:24.411339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=66 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:35.015455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b1d5d-801f-4f35-b9d3-0d67b5d71660 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:35.016023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b1d5d-801f-4f35-b9d3-0d67b5d71660 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:35.016127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b1d5d-801f-4f35-b9d3-0d67b5d71660 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:35.017446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 753/1471] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:35.022738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703696a6-8c90-440d-93ae-cc2315944193 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:35.023195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703696a6-8c90-440d-93ae-cc2315944193 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:35.023402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703696a6-8c90-440d-93ae-cc2315944193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:35.092425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-703696a6-8c90-440d-93ae-cc2315944193 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:35.093638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-703696a6-8c90-440d-93ae-cc2315944193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:35.188877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 719/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:35.271774 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6bbae-b04c-4ca3-99b7-739bec7b2a74 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:35.272068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6bbae-b04c-4ca3-99b7-739bec7b2a74 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:35.272349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6bbae-b04c-4ca3-99b7-739bec7b2a74 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:35.274416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 754/1473] 10.222.0.48 () {54 vars in 966 bytes} [Mon Aug 1 07:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:54:35.279149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e7031-34e6-48d2-975d-e045d65ad7c7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:35.279440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e7031-34e6-48d2-975d-e045d65ad7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:35.279746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e7031-34e6-48d2-975d-e045d65ad7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:35.355793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-252e7031-34e6-48d2-975d-e045d65ad7c7 None None] MFA Rules not processed for user `2f2160adab864eb9970b7767eeeb4789`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:35.357001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-252e7031-34e6-48d2-975d-e045d65ad7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:35.376813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:35.431691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 720/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Mon Aug 1 07:54:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:54:35.439613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6bbae-b04c-4ca3-99b7-739bec7b2a74 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:35.441218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dbd6bbae-b04c-4ca3-99b7-739bec7b2a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:35.545504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6bbae-b04c-4ca3-99b7-739bec7b2a74 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:35.621505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a899055-cd4b-4851-9c5b-ce35fcfe7d5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:35.622134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a899055-cd4b-4851-9c5b-ce35fcfe7d5b service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:35.622415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a899055-cd4b-4851-9c5b-ce35fcfe7d5b service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:35.622716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a899055-cd4b-4851-9c5b-ce35fcfe7d5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:35.624402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5a899055-cd4b-4851-9c5b-ce35fcfe7d5b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:35.679037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:35.735001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a899055-cd4b-4851-9c5b-ce35fcfe7d5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:35.736576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a899055-cd4b-4851-9c5b-ce35fcfe7d5b service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:35.754924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 755/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:37.841153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaac27-876d-498f-9ae6-77fbe80e70df None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:37.841727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaac27-876d-498f-9ae6-77fbe80e70df None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:37.841727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaac27-876d-498f-9ae6-77fbe80e70df None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:37.843116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 721/1476] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:54:37.848187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8df7-4a6e-4147-8298-a3b7664895f1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:37.848637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8df7-4a6e-4147-8298-a3b7664895f1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:37.848754 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8df7-4a6e-4147-8298-a3b7664895f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:37.916618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-01bf8df7-4a6e-4147-8298-a3b7664895f1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:37.917875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-01bf8df7-4a6e-4147-8298-a3b7664895f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:38.016057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 756/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:38.038951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaac27-876d-498f-9ae6-77fbe80e70df None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:38.040724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8ffaac27-876d-498f-9ae6-77fbe80e70df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:38.166576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaac27-876d-498f-9ae6-77fbe80e70df None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:38.215024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:38.261777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151eaf-51e1-4e2e-aa11-b671b0d55a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:38.262400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151eaf-51e1-4e2e-aa11-b671b0d55a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:38.262715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151eaf-51e1-4e2e-aa11-b671b0d55a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:38.262993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151eaf-51e1-4e2e-aa11-b671b0d55a2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:38.263722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a151eaf-51e1-4e2e-aa11-b671b0d55a2d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:38.265455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-0a151eaf-51e1-4e2e-aa11-b671b0d55a2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:54:38.265455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 07:54:38.265455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 07:54:38.266645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a151eaf-51e1-4e2e-aa11-b671b0d55a2d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:38.285657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 722/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 07:54:38 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:54:40.305252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419a6f-63a4-46fc-bc62-b06f4aa25803 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:40.305837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419a6f-63a4-46fc-bc62-b06f4aa25803 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:40.305837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419a6f-63a4-46fc-bc62-b06f4aa25803 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:40.307209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 757/1479] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:40.312487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d93556-782d-41f3-809f-5d3c6bd5f49d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:40.312903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d93556-782d-41f3-809f-5d3c6bd5f49d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:40.313044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d93556-782d-41f3-809f-5d3c6bd5f49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:40.379327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a2d93556-782d-41f3-809f-5d3c6bd5f49d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:40.380573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a2d93556-782d-41f3-809f-5d3c6bd5f49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:40.476485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 723/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:40.519994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab419a6f-63a4-46fc-bc62-b06f4aa25803 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:40.521535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ab419a6f-63a4-46fc-bc62-b06f4aa25803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:40.531782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:40.671064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab419a6f-63a4-46fc-bc62-b06f4aa25803 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:40.750862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:40.786991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93d565-2af3-429a-9afc-a2ef3bc7a879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:40.787633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93d565-2af3-429a-9afc-a2ef3bc7a879 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:40.787916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93d565-2af3-429a-9afc-a2ef3bc7a879 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:40.788187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93d565-2af3-429a-9afc-a2ef3bc7a879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:40.790064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5c93d565-2af3-429a-9afc-a2ef3bc7a879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:40.877565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93d565-2af3-429a-9afc-a2ef3bc7a879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:40.879178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93d565-2af3-429a-9afc-a2ef3bc7a879 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:40.897448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 758/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:43.072202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bb6a-cfdf-478e-b29f-3484c5c26e95 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:43.072719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bb6a-cfdf-478e-b29f-3484c5c26e95 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:43.072719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bb6a-cfdf-478e-b29f-3484c5c26e95 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:43.074023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 724/1482] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:43.079295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cd988-25cb-4049-ba52-9b363c7b84c8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:43.079747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cd988-25cb-4049-ba52-9b363c7b84c8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:43.079894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cd988-25cb-4049-ba52-9b363c7b84c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:43.146754 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-721cd988-25cb-4049-ba52-9b363c7b84c8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:43.148080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-721cd988-25cb-4049-ba52-9b363c7b84c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:43.194052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:43.244398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 759/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:43.280148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384bb6a-cfdf-478e-b29f-3484c5c26e95 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:43.281771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7384bb6a-cfdf-478e-b29f-3484c5c26e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:43.306632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:43.387037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384bb6a-cfdf-478e-b29f-3484c5c26e95 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:43.460661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28870c5-5620-443f-85c8-2dae8f8857a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:43.461284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28870c5-5620-443f-85c8-2dae8f8857a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:43.461571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28870c5-5620-443f-85c8-2dae8f8857a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:43.461849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28870c5-5620-443f-85c8-2dae8f8857a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:43.463502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c28870c5-5620-443f-85c8-2dae8f8857a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:43.537536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:43.574726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28870c5-5620-443f-85c8-2dae8f8857a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:43.576196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28870c5-5620-443f-85c8-2dae8f8857a4 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:43.594782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 725/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:45.641094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f6c2-0cca-4b56-9af3-4d4f736d824e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:45.641655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f6c2-0cca-4b56-9af3-4d4f736d824e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:45.641655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f6c2-0cca-4b56-9af3-4d4f736d824e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:45.643016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 760/1485] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:45.648251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b01d1-d5ef-493e-b0a1-5b8718607167 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:45.648701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b01d1-d5ef-493e-b0a1-5b8718607167 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:45.648811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b01d1-d5ef-493e-b0a1-5b8718607167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:45.714647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bb1b01d1-d5ef-493e-b0a1-5b8718607167 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:45.715914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bb1b01d1-d5ef-493e-b0a1-5b8718607167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:45.814536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 726/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:45.854407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f6c2-0cca-4b56-9af3-4d4f736d824e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:45.856084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2142f6c2-0cca-4b56-9af3-4d4f736d824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:45.958059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f6c2-0cca-4b56-9af3-4d4f736d824e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:46.028963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:46.030574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e2d08-cb4d-49fe-aaf9-e7937113f569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:46.031513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e2d08-cb4d-49fe-aaf9-e7937113f569 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:46.031833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e2d08-cb4d-49fe-aaf9-e7937113f569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:46.032119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e2d08-cb4d-49fe-aaf9-e7937113f569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:46.033943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b62e2d08-cb4d-49fe-aaf9-e7937113f569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:46.147110 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e2d08-cb4d-49fe-aaf9-e7937113f569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:46.149406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e2d08-cb4d-49fe-aaf9-e7937113f569 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:46.168812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 761/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:48.308159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d588c1f-4602-4773-a14e-649f47665814 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:48.308706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d588c1f-4602-4773-a14e-649f47665814 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:48.308821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d588c1f-4602-4773-a14e-649f47665814 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:48.310129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 727/1488] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:48.315456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1356f8-421b-4344-bbdf-124bca9346db None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:48.315892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1356f8-421b-4344-bbdf-124bca9346db None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:48.316058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1356f8-421b-4344-bbdf-124bca9346db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:48.385681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0b1356f8-421b-4344-bbdf-124bca9346db None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:48.386902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0b1356f8-421b-4344-bbdf-124bca9346db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:48.442117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:48.486164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 762/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:48.526757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d588c1f-4602-4773-a14e-649f47665814 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:48.528388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7d588c1f-4602-4773-a14e-649f47665814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:48.565291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:48.633573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d588c1f-4602-4773-a14e-649f47665814 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:48.707653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb15773-21fb-4307-8dc1-32409e1985e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:48.708278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15773-21fb-4307-8dc1-32409e1985e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:48.708562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15773-21fb-4307-8dc1-32409e1985e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:48.708837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15773-21fb-4307-8dc1-32409e1985e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:48.710705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-edb15773-21fb-4307-8dc1-32409e1985e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:48.793847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:48.823026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb15773-21fb-4307-8dc1-32409e1985e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:48.824432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb15773-21fb-4307-8dc1-32409e1985e3 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:48.843125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 728/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:54:51.159418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148501b-5778-41fe-91e4-2706ba694f89 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:51.160052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148501b-5778-41fe-91e4-2706ba694f89 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:51.160052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148501b-5778-41fe-91e4-2706ba694f89 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:51.161379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 763/1491] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 07:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:54:51.167709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9787a-7d06-4ed4-9acf-c4257910dcd0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:51.168128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9787a-7d06-4ed4-9acf-c4257910dcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:51.168274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9787a-7d06-4ed4-9acf-c4257910dcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:51.236950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-55c9787a-7d06-4ed4-9acf-c4257910dcd0 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:54:51.238176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-55c9787a-7d06-4ed4-9acf-c4257910dcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:51.335706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 729/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 07:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:54:51.375137 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148501b-5778-41fe-91e4-2706ba694f89 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:54:51.376726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a148501b-5778-41fe-91e4-2706ba694f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:51.480702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148501b-5778-41fe-91e4-2706ba694f89 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:54:51.556544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ca920-124c-43da-830b-6ce878b59272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:54:51.557170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ca920-124c-43da-830b-6ce878b59272 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:54:51.557457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ca920-124c-43da-830b-6ce878b59272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:54:51.557733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ca920-124c-43da-830b-6ce878b59272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:54:51.559443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ed1ca920-124c-43da-830b-6ce878b59272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:54:51.569485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:54:51.670329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ca920-124c-43da-830b-6ce878b59272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:54:51.671805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ca920-124c-43da-830b-6ce878b59272 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:54:51.689707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 764/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:55:01.333268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=82 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:55:01.687417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=50 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:27.576989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:27.577458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:27.577573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:27.653988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:56:27.655217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:27.753079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 730/1494] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 07:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:56:27.822685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8fe02-06fd-47fd-afda-b2e776d570c8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:27.822685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8fe02-06fd-47fd-afda-b2e776d570c8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:27.822685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8fe02-06fd-47fd-afda-b2e776d570c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:27.842292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f5e8fe02-06fd-47fd-afda-b2e776d570c8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:56:27.843516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f5e8fe02-06fd-47fd-afda-b2e776d570c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:27.940489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 765/1495] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 07:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:56:27.945050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:27.949568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:27.957732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e4aa6-ede3-41a4-a049-4e961291d652 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:28.019626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:28.053487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:28.054112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:28.054396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:28.054673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:28.055368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:56:28.055708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:28.056725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:28.056725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 07:56:28.056725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 07:56:28.057622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:28.069548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 731/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 07:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:56:28.077959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8fe02-06fd-47fd-afda-b2e776d570c8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:28.087416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8fe02-06fd-47fd-afda-b2e776d570c8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:28.148738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:28.182835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:28.183765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:28.184324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:28.184925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:28.185612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:28.187646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:28.187646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: warnings.warn(msg) Aug 01 07:56:28.187646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: Aug 01 07:56:28.189053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:28.316893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 766/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 07:56:28 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:56:28.321085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:28.322609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:28.355960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af2299-5e48-4837-afe2-ae847d553223 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:28.452633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:28.453278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:28.453563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:28.453836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:28.454543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:28.455590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:28.455590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 07:56:28.455590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 07:56:28.456809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:28.555274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 732/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 07:56:28 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:56:28.559337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:28.560940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:28.622568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07965b8d-3674-4272-a12a-ec3e20740284 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:28.705897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:28.716291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:28.716912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:28.717219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:28.717492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:28.718190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:28.719878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:28.719878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: warnings.warn(msg) Aug 01 07:56:28.719878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: Aug 01 07:56:28.721353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:28.734055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 767/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 07:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:56:28.738203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:28.739779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:28.751791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd8009-1777-41f4-b6f5-05163477f45d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:28.777519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:28.847231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e0479-63be-436f-b84f-afeaf0f0ec04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:28.847881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e0479-63be-436f-b84f-afeaf0f0ec04 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:28.848161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e0479-63be-436f-b84f-afeaf0f0ec04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:28.848429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e0479-63be-436f-b84f-afeaf0f0ec04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:28.849116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e0479-63be-436f-b84f-afeaf0f0ec04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:28.850130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-365e0479-63be-436f-b84f-afeaf0f0ec04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:28.850130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 07:56:28.850130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 07:56:28.851010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e0479-63be-436f-b84f-afeaf0f0ec04 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:28.863274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 733/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 07:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:56:28.867512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:28.869101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:28.904294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6aee-9990-49ea-a93c-40ab1b887aa5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:28.996865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c618f1-c789-4ccd-8566-0cdeb6186764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:28.997530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c618f1-c789-4ccd-8566-0cdeb6186764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:28.997814 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c618f1-c789-4ccd-8566-0cdeb6186764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:28.998090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c618f1-c789-4ccd-8566-0cdeb6186764 admin admin] PATH_INFO: `/v3/projects/a63747ddbb4e40a0a37da943817c4e96/users/aefe81a75b62431886379aecc84969ca/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:29.020618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c618f1-c789-4ccd-8566-0cdeb6186764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63747ddbb4e40a0a37da943817c4e96, user_id=aefe81a75b62431886379aecc84969ca, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:29.021652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-f2c618f1-c789-4ccd-8566-0cdeb6186764 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:29.021652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: warnings.warn(msg) Aug 01 07:56:29.021652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: Aug 01 07:56:29.023612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c618f1-c789-4ccd-8566-0cdeb6186764 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:29.150259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 768/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 07:56:28 2022] PUT /identity/v3/projects/a63747ddbb4e40a0a37da943817c4e96/users/aefe81a75b62431886379aecc84969ca/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:56:29.162621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a6293-d77b-4b1b-96df-2ffc55bc11d1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:29.162853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a6293-d77b-4b1b-96df-2ffc55bc11d1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:29.163132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a6293-d77b-4b1b-96df-2ffc55bc11d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:29.220259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2e1a6293-d77b-4b1b-96df-2ffc55bc11d1 None None] MFA Rules not processed for user `aefe81a75b62431886379aecc84969ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:56:29.221505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2e1a6293-d77b-4b1b-96df-2ffc55bc11d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:29.299064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 734/1502] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 07:56:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:56:29.312309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa2612-ea28-4b24-be2e-746d1b845dc1 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:29.313259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa2612-ea28-4b24-be2e-746d1b845dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:29.313540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa2612-ea28-4b24-be2e-746d1b845dc1 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:29.314894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 769/1503] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 1 07:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:56:29.319786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be244b1-596d-4942-ae91-f2ffdc1eb5b3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:29.320077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be244b1-596d-4942-ae91-f2ffdc1eb5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:29.320353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be244b1-596d-4942-ae91-f2ffdc1eb5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:29.350637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:29.395175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2be244b1-596d-4942-ae91-f2ffdc1eb5b3 None None] MFA Rules not processed for user `317df4b0baa54808af9b7918ff5c3515`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:56:29.396398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2be244b1-596d-4942-ae91-f2ffdc1eb5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:29.470482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 735/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 1 07:56:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:56:29.478113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa2612-ea28-4b24-be2e-746d1b845dc1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:29.479695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8aaa2612-ea28-4b24-be2e-746d1b845dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:29.530109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:29.585735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa2612-ea28-4b24-be2e-746d1b845dc1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:29.660445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:29.661101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:29.661418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:29.661698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:29.663384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:29.761669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:29.763155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:29.767056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:29.781926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 770/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:56:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:56:29.827962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18f42-3676-44f9-b8cd-6675e69cd498 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:29.828593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18f42-3676-44f9-b8cd-6675e69cd498 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:29.828593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18f42-3676-44f9-b8cd-6675e69cd498 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:29.829867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 736/1506] 10.222.0.48 () {52 vars in 830 bytes} [Mon Aug 1 07:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:56:29.833478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:29.834694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:29.934290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2b117-5ddd-449d-9f0d-1acadb3d2328 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:30.030004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:30.030560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:30.030841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:30.031114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] PATH_INFO: `/v3/users/aefe81a75b62431886379aecc84969ca` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:30.043631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefe81a75b62431886379aecc84969ca)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:30.044668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:30.044668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: warnings.warn(msg) Aug 01 07:56:30.044668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: Aug 01 07:56:30.045869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:30.055616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:30.260292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefe81a75b62431886379aecc84969ca'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:30.331745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefe81a75b62431886379aecc84969ca'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:30.346521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefe81a75b62431886379aecc84969ca'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:30.347015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefe81a75b62431886379aecc84969ca'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:30.347570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefe81a75b62431886379aecc84969ca'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:30.377736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 771/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 07:56:29 2022] DELETE /identity/v3/users/aefe81a75b62431886379aecc84969ca => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:56:30.382338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:30.382636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:30.383052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:30.441991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:56:30.443195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:30.520000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:30.532786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 737/1508] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 07:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:56:30.543019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:30.544329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:30.644143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2751-4364-40c2-9c20-cb4d89576862 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:30.699496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:30.719012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198b0d3-5b55-47bd-9de2-7e948112e2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:30.719708 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b0d3-5b55-47bd-9de2-7e948112e2cb service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:30.719997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b0d3-5b55-47bd-9de2-7e948112e2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:30.720276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b0d3-5b55-47bd-9de2-7e948112e2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:30.721930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3198b0d3-5b55-47bd-9de2-7e948112e2cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:30.835988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198b0d3-5b55-47bd-9de2-7e948112e2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:30.837448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198b0d3-5b55-47bd-9de2-7e948112e2cb service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:30.855806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 772/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:56:31.346470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:56:31.353784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:56:31.360054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a90631-f9d5-43be-9fc0-9d5efac5a107 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:56:31.460276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:56:31.460996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:56:31.461283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:56:31.461559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] PATH_INFO: `/v3/projects/a63747ddbb4e40a0a37da943817c4e96` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:56:31.472113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63747ddbb4e40a0a37da943817c4e96)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:56:31.473138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:56:31.473138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 07:56:31.473138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 07:56:31.474445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:56:31.611063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63747ddbb4e40a0a37da943817c4e96', 'request_id': 'req-fdf6c4a4-1ded-491c-8124-ff636d2ec189'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:31.624280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63747ddbb4e40a0a37da943817c4e96', 'request_id': 'req-fdf6c4a4-1ded-491c-8124-ff636d2ec189'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:31.631268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63747ddbb4e40a0a37da943817c4e96', 'request_id': 'req-fdf6c4a4-1ded-491c-8124-ff636d2ec189'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:31.725933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] The token cache is being invalidate because project a63747ddbb4e40a0a37da943817c4e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:56:31.726224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:31.726747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fdf6c4a4-1ded-491c-8124-ff636d2ec189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:56:31.728182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 738/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 07:56:31 2022] DELETE /identity/v3/projects/a63747ddbb4e40a0a37da943817c4e96 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:56:40.853535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=62 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:56:41.723798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:58:07.839292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14de4f-892c-45f9-a7f7-360b6dc45266 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:07.839909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14de4f-892c-45f9-a7f7-360b6dc45266 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:07.840031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14de4f-892c-45f9-a7f7-360b6dc45266 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:07.841382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 773/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:07.846858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7ff61-744c-4579-9285-6e6120c1ec82 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:07.847305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7ff61-744c-4579-9285-6e6120c1ec82 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:07.847403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7ff61-744c-4579-9285-6e6120c1ec82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:07.933694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bcd7ff61-744c-4579-9285-6e6120c1ec82 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:07.934928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bcd7ff61-744c-4579-9285-6e6120c1ec82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:08.040123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 739/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:08.194841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14de4f-892c-45f9-a7f7-360b6dc45266 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:08.197069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5a14de4f-892c-45f9-a7f7-360b6dc45266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:08.317116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14de4f-892c-45f9-a7f7-360b6dc45266 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:08.322597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162c653-7dc6-495a-a068-91a342b42cdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:08.323208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c653-7dc6-495a-a068-91a342b42cdc service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:08.323489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c653-7dc6-495a-a068-91a342b42cdc service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:08.323794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c653-7dc6-495a-a068-91a342b42cdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:08.332497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162c653-7dc6-495a-a068-91a342b42cdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:08.337861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162c653-7dc6-495a-a068-91a342b42cdc service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:08.343807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 774/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:09.179213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee255446-0a01-4a2f-b3bb-ded40a624dad None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:09.179809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee255446-0a01-4a2f-b3bb-ded40a624dad None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:09.179809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee255446-0a01-4a2f-b3bb-ded40a624dad None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:09.181164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 740/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:09.186297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fd995-cf86-4091-aa00-3728b659aefc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:09.186741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fd995-cf86-4091-aa00-3728b659aefc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:09.186849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fd995-cf86-4091-aa00-3728b659aefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:09.206991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b50fd995-cf86-4091-aa00-3728b659aefc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:09.208256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b50fd995-cf86-4091-aa00-3728b659aefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:09.216227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 775/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:09.368278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee255446-0a01-4a2f-b3bb-ded40a624dad None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:09.382556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee255446-0a01-4a2f-b3bb-ded40a624dad None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:09.388032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69aca4-d782-4fe4-a4fa-237b4aa76322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:09.388638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69aca4-d782-4fe4-a4fa-237b4aa76322 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:09.388923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69aca4-d782-4fe4-a4fa-237b4aa76322 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:09.389197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69aca4-d782-4fe4-a4fa-237b4aa76322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:09.397518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69aca4-d782-4fe4-a4fa-237b4aa76322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:09.399097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69aca4-d782-4fe4-a4fa-237b4aa76322 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:09.408919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 741/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:10.447035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174af33-03e0-4a64-9375-b1be864737f8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:10.447624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174af33-03e0-4a64-9375-b1be864737f8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:10.447624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174af33-03e0-4a64-9375-b1be864737f8 None None] PATH_INFO: `/v3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:10.448924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 776/1517] 10.222.0.46 () {54 vars in 925 bytes} [Mon Aug 1 07:58:10 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:58:10.455759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78b45-6674-4985-b689-04cf7128733e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:10.456214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78b45-6674-4985-b689-04cf7128733e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:10.456340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78b45-6674-4985-b689-04cf7128733e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:10.515054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e7b78b45-6674-4985-b689-04cf7128733e None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:10.516284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e7b78b45-6674-4985-b689-04cf7128733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:10.533124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:58:10.587848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 742/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Aug 1 07:58:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:14.623334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f447-005d-4ea1-8bb3-f0b7b9705b1e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:14.623894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f447-005d-4ea1-8bb3-f0b7b9705b1e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:14.623894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f447-005d-4ea1-8bb3-f0b7b9705b1e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:14.625188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 777/1519] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 1 07:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:14.630228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6d4b-cdff-4350-9ccd-405883a8c9ed None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:14.630783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6d4b-cdff-4350-9ccd-405883a8c9ed None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:14.630783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6d4b-cdff-4350-9ccd-405883a8c9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:14.649970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-200f6d4b-cdff-4350-9ccd-405883a8c9ed None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:14.651186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-200f6d4b-cdff-4350-9ccd-405883a8c9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:14.660284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 743/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 1 07:58:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:14.667876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807f447-005d-4ea1-8bb3-f0b7b9705b1e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:14.682423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807f447-005d-4ea1-8bb3-f0b7b9705b1e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:14.687446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d24d2-3d05-45a1-a130-72a01f02c2a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:14.688090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d24d2-3d05-45a1-a130-72a01f02c2a3 service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:14.688380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d24d2-3d05-45a1-a130-72a01f02c2a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:14.688660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d24d2-3d05-45a1-a130-72a01f02c2a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:14.697190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d24d2-3d05-45a1-a130-72a01f02c2a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:14.698777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d24d2-3d05-45a1-a130-72a01f02c2a3 service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:14.708739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 778/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:20.634225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07b8a-65a5-4a63-9047-d97e4a11c96e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:20.634810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07b8a-65a5-4a63-9047-d97e4a11c96e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:20.634810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07b8a-65a5-4a63-9047-d97e4a11c96e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:20.636180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 744/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:20.640987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dc2d2-ad46-4b3d-abb0-363a08b61e7e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:20.641413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dc2d2-ad46-4b3d-abb0-363a08b61e7e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:20.641573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dc2d2-ad46-4b3d-abb0-363a08b61e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:20.662405 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-822dc2d2-ad46-4b3d-abb0-363a08b61e7e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:20.663818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-822dc2d2-ad46-4b3d-abb0-363a08b61e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:20.671654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 779/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:20.826840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c07b8a-65a5-4a63-9047-d97e4a11c96e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:20.841764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c07b8a-65a5-4a63-9047-d97e4a11c96e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:20.849226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bbcde-ddf3-4f52-8c92-958901b56658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:20.849226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bbcde-ddf3-4f52-8c92-958901b56658 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:20.849226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bbcde-ddf3-4f52-8c92-958901b56658 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:20.849568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bbcde-ddf3-4f52-8c92-958901b56658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:20.857849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bbcde-ddf3-4f52-8c92-958901b56658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:20.859429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bbcde-ddf3-4f52-8c92-958901b56658 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:20.869133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 745/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:21.707434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c2887-03a8-4f46-af01-eb28adf6828f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:21.708029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c2887-03a8-4f46-af01-eb28adf6828f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:21.708029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c2887-03a8-4f46-af01-eb28adf6828f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:21.709439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 780/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:21.714358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca2ecb-56e7-41b5-a394-c7f8efba64d6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:21.714812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca2ecb-56e7-41b5-a394-c7f8efba64d6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:21.714929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca2ecb-56e7-41b5-a394-c7f8efba64d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:21.734241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4eca2ecb-56e7-41b5-a394-c7f8efba64d6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:21.735448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4eca2ecb-56e7-41b5-a394-c7f8efba64d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:21.744079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 746/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:21.895820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c2887-03a8-4f46-af01-eb28adf6828f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:21.910260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c2887-03a8-4f46-af01-eb28adf6828f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:21.915707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c60b45-dbc8-4299-93fd-21c16444a0f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:21.916312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60b45-dbc8-4299-93fd-21c16444a0f3 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:21.916596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60b45-dbc8-4299-93fd-21c16444a0f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:21.916873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60b45-dbc8-4299-93fd-21c16444a0f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:21.925436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c60b45-dbc8-4299-93fd-21c16444a0f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:21.927135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c60b45-dbc8-4299-93fd-21c16444a0f3 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:21.936753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 781/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:25.757857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf46dc3-e0fe-415d-b739-268a859227fb None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:25.758398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf46dc3-e0fe-415d-b739-268a859227fb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:25.758398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf46dc3-e0fe-415d-b739-268a859227fb None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:25.759757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 747/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:25.764921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3627bf6-8835-4292-835d-19d604ffa385 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:25.765320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3627bf6-8835-4292-835d-19d604ffa385 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:25.765487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3627bf6-8835-4292-835d-19d604ffa385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:25.785679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b3627bf6-8835-4292-835d-19d604ffa385 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:25.786960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b3627bf6-8835-4292-835d-19d604ffa385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:25.795094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 782/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:25.868235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf46dc3-e0fe-415d-b739-268a859227fb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:25.869970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3cf46dc3-e0fe-415d-b739-268a859227fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:25.948038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf46dc3-e0fe-415d-b739-268a859227fb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:25.951590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff206d-b1b3-44fa-84a9-2ae8a06d7328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:25.952207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff206d-b1b3-44fa-84a9-2ae8a06d7328 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:25.952492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff206d-b1b3-44fa-84a9-2ae8a06d7328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:25.952769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff206d-b1b3-44fa-84a9-2ae8a06d7328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:25.962323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff206d-b1b3-44fa-84a9-2ae8a06d7328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:25.963780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff206d-b1b3-44fa-84a9-2ae8a06d7328 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:25.973187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 748/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:27.966792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87c4d-4347-4f55-aafe-58256f49e0de None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:27.967364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87c4d-4347-4f55-aafe-58256f49e0de None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:27.967364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87c4d-4347-4f55-aafe-58256f49e0de None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:27.968752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 783/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:27.974671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7629ca-65fd-4369-862e-0faa7de922b0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:27.978513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7629ca-65fd-4369-862e-0faa7de922b0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:27.978513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7629ca-65fd-4369-862e-0faa7de922b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:27.995750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2b7629ca-65fd-4369-862e-0faa7de922b0 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:27.996972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2b7629ca-65fd-4369-862e-0faa7de922b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:28.005188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 749/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:28.040790 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87c4d-4347-4f55-aafe-58256f49e0de None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:28.052331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87c4d-4347-4f55-aafe-58256f49e0de None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:28.056131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95852e15-90d4-4583-8716-6ec005409a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:28.056731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852e15-90d4-4583-8716-6ec005409a71 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:28.057013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852e15-90d4-4583-8716-6ec005409a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:28.057290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852e15-90d4-4583-8716-6ec005409a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:28.066290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95852e15-90d4-4583-8716-6ec005409a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:28.067844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95852e15-90d4-4583-8716-6ec005409a71 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:28.077115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 784/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:31.504738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618d9c9-134d-4c0d-bc6e-4ba8e32d8475 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:31.505320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618d9c9-134d-4c0d-bc6e-4ba8e32d8475 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:31.505320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618d9c9-134d-4c0d-bc6e-4ba8e32d8475 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:31.506682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 750/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:31.512013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d992-5073-4c0a-b19d-e5cad07f0b20 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:31.515736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d992-5073-4c0a-b19d-e5cad07f0b20 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:31.515736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d992-5073-4c0a-b19d-e5cad07f0b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:31.520543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=81 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:58:31.532673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d898d992-5073-4c0a-b19d-e5cad07f0b20 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:31.533887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d898d992-5073-4c0a-b19d-e5cad07f0b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:31.541778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 785/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:31.848700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618d9c9-134d-4c0d-bc6e-4ba8e32d8475 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:31.850384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4618d9c9-134d-4c0d-bc6e-4ba8e32d8475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:31.916464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:58:31.929712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618d9c9-134d-4c0d-bc6e-4ba8e32d8475 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:31.933579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4d2d3-e958-424e-8f74-265a03767fcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:31.934193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d2d3-e958-424e-8f74-265a03767fcb service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:31.934472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d2d3-e958-424e-8f74-265a03767fcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:31.934742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d2d3-e958-424e-8f74-265a03767fcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:31.944506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4d2d3-e958-424e-8f74-265a03767fcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:31.945920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4d2d3-e958-424e-8f74-265a03767fcb service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:31.956020 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 751/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:33.957721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc38760-3394-4d8b-b015-18bbb2b80128 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:33.958283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc38760-3394-4d8b-b015-18bbb2b80128 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:33.958283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc38760-3394-4d8b-b015-18bbb2b80128 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:33.959536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 786/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:58:33.964678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6c335-68f2-4077-a281-52ae9967a58c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:33.965094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6c335-68f2-4077-a281-52ae9967a58c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:33.965240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6c335-68f2-4077-a281-52ae9967a58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:33.984789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5eb6c335-68f2-4077-a281-52ae9967a58c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:58:33.986038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5eb6c335-68f2-4077-a281-52ae9967a58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:33.994261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 752/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:58:34.299986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc38760-3394-4d8b-b015-18bbb2b80128 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:58:34.301798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-edc38760-3394-4d8b-b015-18bbb2b80128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:58:34.315782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc38760-3394-4d8b-b015-18bbb2b80128 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:58:34.325450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2a30c-7151-4916-88ae-d4a047d34893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:58:34.325450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2a30c-7151-4916-88ae-d4a047d34893 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:58:34.325450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2a30c-7151-4916-88ae-d4a047d34893 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:58:34.325450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2a30c-7151-4916-88ae-d4a047d34893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:58:34.329310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2a30c-7151-4916-88ae-d4a047d34893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:58:34.330903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2a30c-7151-4916-88ae-d4a047d34893 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:58:34.340312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 787/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 07:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:58:43.979903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=11 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:58:44.336185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=8 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:02.868340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22088c9d-6b90-41ce-9c1d-0d02de2f46fc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:02.869830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22088c9d-6b90-41ce-9c1d-0d02de2f46fc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:02.869830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22088c9d-6b90-41ce-9c1d-0d02de2f46fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:02.895257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-22088c9d-6b90-41ce-9c1d-0d02de2f46fc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:02.896862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-22088c9d-6b90-41ce-9c1d-0d02de2f46fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:02.909766 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 753/1540] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:02.918776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83819f1b-b2da-47de-9c51-de706f1c3c9a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:02.919316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83819f1b-b2da-47de-9c51-de706f1c3c9a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:02.919316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83819f1b-b2da-47de-9c51-de706f1c3c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:02.948065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-83819f1b-b2da-47de-9c51-de706f1c3c9a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:02.949302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-83819f1b-b2da-47de-9c51-de706f1c3c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:02.963667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 788/1541] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:02.969011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22088c9d-6b90-41ce-9c1d-0d02de2f46fc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:02.984855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22088c9d-6b90-41ce-9c1d-0d02de2f46fc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:02.993473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:02.994667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:02.995184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:02.995697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:02.996618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:02.997147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:02.999025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.015747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e44ee-815c-4a7a-bdb4-92689943993d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.015956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e44ee-815c-4a7a-bdb4-92689943993d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.016231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e44ee-815c-4a7a-bdb4-92689943993d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.021803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 754/1542] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:03.029854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.041651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-845e44ee-815c-4a7a-bdb4-92689943993d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.042880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-845e44ee-815c-4a7a-bdb4-92689943993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.045848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152fd9-94c9-409c-a200-153fdf0858e5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.055022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc68f8-0121-4b3c-842e-b7faa0b07843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.056187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc68f8-0121-4b3c-842e-b7faa0b07843 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.056450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 789/1543] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.056913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc68f8-0121-4b3c-842e-b7faa0b07843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.057399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc68f8-0121-4b3c-842e-b7faa0b07843 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.058292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc68f8-0121-4b3c-842e-b7faa0b07843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.059578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-befc68f8-0121-4b3c-842e-b7faa0b07843 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 08:00:03.059578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 08:00:03.059578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 08:00:03.061309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc68f8-0121-4b3c-842e-b7faa0b07843 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.065674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816038-2a52-4bb8-954a-e368256417de None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.065836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816038-2a52-4bb8-954a-e368256417de None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.066085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816038-2a52-4bb8-954a-e368256417de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.087383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4c816038-2a52-4bb8-954a-e368256417de None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.088619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4c816038-2a52-4bb8-954a-e368256417de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.107453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 790/1544] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:03.108880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480761fa-4ab6-4d2f-802c-04c72829735b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.109172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480761fa-4ab6-4d2f-802c-04c72829735b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.109574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480761fa-4ab6-4d2f-802c-04c72829735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.149619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-480761fa-4ab6-4d2f-802c-04c72829735b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.151174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-480761fa-4ab6-4d2f-802c-04c72829735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.164858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 791/1545] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.165701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480761fa-4ab6-4d2f-802c-04c72829735b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.179743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480761fa-4ab6-4d2f-802c-04c72829735b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.187213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.188415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.188978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.189634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.190578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:03.191205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.192601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 08:00:03.192601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: warnings.warn(msg) Aug 01 08:00:03.192601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: Aug 01 08:00:03.193830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc535c0-7cd1-45bf-9ff3-24c0d49c8451 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.213419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 755/1546] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:00:03.215694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b66663-9674-4dde-a18a-c1a891c4c55b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.215694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b66663-9674-4dde-a18a-c1a891c4c55b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.215694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b66663-9674-4dde-a18a-c1a891c4c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.220401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 792/1547] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:03.222462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0e42-3ef2-488e-b928-e2f2fec03fc8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.223078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0e42-3ef2-488e-b928-e2f2fec03fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.223609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0e42-3ef2-488e-b928-e2f2fec03fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.241100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f3b66663-9674-4dde-a18a-c1a891c4c55b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.242659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f3b66663-9674-4dde-a18a-c1a891c4c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.246841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-189f0e42-3ef2-488e-b928-e2f2fec03fc8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.248989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-189f0e42-3ef2-488e-b928-e2f2fec03fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.306466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 756/1548] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.308294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84637db8-9bd5-4d89-9f63-a61ab0a52a1c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.308613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84637db8-9bd5-4d89-9f63-a61ab0a52a1c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.308958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84637db8-9bd5-4d89-9f63-a61ab0a52a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.324821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 793/1549] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.326621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181d72c-115a-4650-b091-ce6b4015bca3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.327683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181d72c-115a-4650-b091-ce6b4015bca3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.328514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181d72c-115a-4650-b091-ce6b4015bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.334045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-84637db8-9bd5-4d89-9f63-a61ab0a52a1c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.335248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-84637db8-9bd5-4d89-9f63-a61ab0a52a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.344514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 757/1550] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.345576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b0c3-643f-4637-aa9c-e2a5257bb455 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.345860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b0c3-643f-4637-aa9c-e2a5257bb455 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.346127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b0c3-643f-4637-aa9c-e2a5257bb455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.349542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3181d72c-115a-4650-b091-ce6b4015bca3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.350757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3181d72c-115a-4650-b091-ce6b4015bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.360595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 794/1551] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.361421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181d72c-115a-4650-b091-ce6b4015bca3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.365318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e709b0c3-643f-4637-aa9c-e2a5257bb455 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.366611 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e709b0c3-643f-4637-aa9c-e2a5257bb455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.371904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181d72c-115a-4650-b091-ce6b4015bca3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.374929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 758/1552] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.375815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709b0c3-643f-4637-aa9c-e2a5257bb455 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.376817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e571f-2775-4099-a827-a8ceff64090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.377460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e571f-2775-4099-a827-a8ceff64090f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.377903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e571f-2775-4099-a827-a8ceff64090f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.378191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e571f-2775-4099-a827-a8ceff64090f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.378858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e571f-2775-4099-a827-a8ceff64090f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.379914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-550e571f-2775-4099-a827-a8ceff64090f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 08:00:03.379914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: warnings.warn(msg) Aug 01 08:00:03.379914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: Aug 01 08:00:03.381363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e571f-2775-4099-a827-a8ceff64090f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.385536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709b0c3-643f-4637-aa9c-e2a5257bb455 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.391061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e7d48-0918-4bcb-ba72-9db58d329267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.391716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e7d48-0918-4bcb-ba72-9db58d329267 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.392008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e7d48-0918-4bcb-ba72-9db58d329267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.392428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e7d48-0918-4bcb-ba72-9db58d329267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.393087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e7d48-0918-4bcb-ba72-9db58d329267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.395004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e7d48-0918-4bcb-ba72-9db58d329267 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.495641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 795/1553] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:03.495641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2c589-0d31-458e-bf0c-1bc7aea0740c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.495641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2c589-0d31-458e-bf0c-1bc7aea0740c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.495641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2c589-0d31-458e-bf0c-1bc7aea0740c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.514415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b9a2c589-0d31-458e-bf0c-1bc7aea0740c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.516420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b9a2c589-0d31-458e-bf0c-1bc7aea0740c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.524654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 796/1554] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.525710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293267df-41b2-42e6-b2fc-284c5e85d1ca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.525998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293267df-41b2-42e6-b2fc-284c5e85d1ca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.526276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293267df-41b2-42e6-b2fc-284c5e85d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.544939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-293267df-41b2-42e6-b2fc-284c5e85d1ca None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.546847 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-293267df-41b2-42e6-b2fc-284c5e85d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.552769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 759/1555] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:03.564016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b67f-5d0a-4ee3-86ce-e8058d5996b3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.564016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b67f-5d0a-4ee3-86ce-e8058d5996b3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.564016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b67f-5d0a-4ee3-86ce-e8058d5996b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.564378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 797/1556] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.564378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293267df-41b2-42e6-b2fc-284c5e85d1ca None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.569918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293267df-41b2-42e6-b2fc-284c5e85d1ca None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.579070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9d20b67f-5d0a-4ee3-86ce-e8058d5996b3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.580395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9d20b67f-5d0a-4ee3-86ce-e8058d5996b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.629040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5f2d7-5b04-4f09-9e25-df2e18046d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.629673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5f2d7-5b04-4f09-9e25-df2e18046d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.629966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5f2d7-5b04-4f09-9e25-df2e18046d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.630248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5f2d7-5b04-4f09-9e25-df2e18046d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.630952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5f2d7-5b04-4f09-9e25-df2e18046d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:03.631264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5f2d7-5b04-4f09-9e25-df2e18046d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.633092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5f2d7-5b04-4f09-9e25-df2e18046d76 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.645817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 760/1557] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.645817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5d8a5-29b0-46ef-9918-762477cf1b44 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.645817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5d8a5-29b0-46ef-9918-762477cf1b44 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.645817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5d8a5-29b0-46ef-9918-762477cf1b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.647757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 798/1558] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:03.649744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9ad8-c8e0-4db9-ae0d-1bd5ec86940c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.650117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9ad8-c8e0-4db9-ae0d-1bd5ec86940c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.650447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9ad8-c8e0-4db9-ae0d-1bd5ec86940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.667554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7ef5d8a5-29b0-46ef-9918-762477cf1b44 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.668777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7ef5d8a5-29b0-46ef-9918-762477cf1b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.673520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b8da9ad8-c8e0-4db9-ae0d-1bd5ec86940c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.674724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b8da9ad8-c8e0-4db9-ae0d-1bd5ec86940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.677769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 761/1559] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.678733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5d8a5-29b0-46ef-9918-762477cf1b44 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.683095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 799/1560] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.683924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da9ad8-c8e0-4db9-ae0d-1bd5ec86940c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.690035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5d8a5-29b0-46ef-9918-762477cf1b44 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.694335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da9ad8-c8e0-4db9-ae0d-1bd5ec86940c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.694843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.695481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.695802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.696168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.696862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.698463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.699833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08fca2-fb70-46ae-8cf1-c09dfb3dd326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.700426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08fca2-fb70-46ae-8cf1-c09dfb3dd326 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.700720 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08fca2-fb70-46ae-8cf1-c09dfb3dd326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.700866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:03.701051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08fca2-fb70-46ae-8cf1-c09dfb3dd326 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.701739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08fca2-fb70-46ae-8cf1-c09dfb3dd326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:03.702702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08fca2-fb70-46ae-8cf1-c09dfb3dd326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.704602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08fca2-fb70-46ae-8cf1-c09dfb3dd326 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.710441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 762/1561] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:03.711747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.712313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:03.717779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 800/1562] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:03.719201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57d52-6c29-4656-aba0-4d105fc3f3e1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.727507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57d52-6c29-4656-aba0-4d105fc3f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.727507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57d52-6c29-4656-aba0-4d105fc3f3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.727753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053468-1a8f-4fad-b0f9-d6efc61a80fb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.731234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.731876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.732160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.732432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.733120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.735366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.743077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-80e57d52-6c29-4656-aba0-4d105fc3f3e1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:03.744322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-80e57d52-6c29-4656-aba0-4d105fc3f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:03.753458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 801/1563] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:03.754332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57d52-6c29-4656-aba0-4d105fc3f3e1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.767416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57d52-6c29-4656-aba0-4d105fc3f3e1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.772391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.773013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.780086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.780086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.780086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:03.780086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.780086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.788464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 802/1564] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:03.789229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.799015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784334-a22a-4c7d-84a8-e5269beb83b9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.803921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.804785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.804902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.805227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.805876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.807779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.859587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 763/1565] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:03.860574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.870276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb8ff-8f46-47c0-b699-8c2c46218a76 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.875779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.876378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.876667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.876945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.877638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:03.877950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.879602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.892078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 764/1566] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:03.892939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.902365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dff985-29d6-4e1f-9a1e-004371ac5ade admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.907291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.907905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.908193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.908594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.909192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:03.909504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.911066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.922919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 765/1567] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:03.923722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.933348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b9619-6a08-4def-8e74-e1f0d11e04dd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:03.938805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:03.939402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:03.939716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:03.940120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:03.940811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:03.942403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:03.946398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 803/1568] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:03.947059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.956430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 766/1569] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:03.956430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:03.963833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34b8ce-8a56-4c47-a39d-5cc92bb8b6f4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.014415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbade8-ee96-4b4e-8ca5-5c6bf87d0d50 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.019406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.020064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.020352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.020634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.021315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.021444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.022084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.022369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.022640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.023249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.023365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:04.023671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.025290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.036960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 767/1570] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:04.037857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.048161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1f8e0-fe79-41e5-9e1e-aeb14b292683 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.053193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.053785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.054179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.054461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.055132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.057018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.106154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 804/1571] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:03 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.107076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.166762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 768/1572] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.166762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.172803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d0314-8974-4eee-bc2e-90e6317e2ca6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.219552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:04.229901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f7da8-9349-4553-9bb1-acc4a0efaba3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.231732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.232360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.232648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.232927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.234472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.235268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.235926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.236118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.236237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.236477 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.237149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.239045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.249635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 805/1573] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:04.249635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.259270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b7260-a68c-41f2-86c9-3abf840b97fc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.264048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.264705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.264992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.265271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.265925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.267870 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.362682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:04.372621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 769/1574] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.372621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.397089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 806/1575] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.399056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.433666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d8be9-8ffe-43d3-b94f-db7dcc49b85b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.461910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405546d-8b8d-425c-a89f-467057ca8e0a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.466644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.467319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.468164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.468444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] PATH_INFO: `/v3/projects/6a3a9bab8b8c4e48b5fc32e1452a591d/users/670637f19e32493181e52e9df6c2f576/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.482911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a3a9bab8b8c4e48b5fc32e1452a591d, user_id=670637f19e32493181e52e9df6c2f576, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.486427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.491662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.492294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.492579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.492856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.493524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.495433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.539292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 807/1576] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:04 2022] PUT /identity/v3/projects/6a3a9bab8b8c4e48b5fc32e1452a591d/users/670637f19e32493181e52e9df6c2f576/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:04.540629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.562808 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 770/1577] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.567960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.573983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18493ea0-d4f6-44a7-8e46-37785e7a01ff admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.613503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e500af-f754-4b5f-9690-cfc40debf398 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.618360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.618986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.619275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.619580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.620250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.622207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.631573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.632274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.632624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.633004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.633765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.635833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.677794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 808/1578] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/projects => generated 391 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.678746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.694579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 771/1579] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.694941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.708064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f901674-2eab-498a-887d-a79bc1afe401 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.715209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:04.734899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:04.746053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362c813-a5e4-4c02-81ed-83e4b5915dda admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.750792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.751426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.751744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.752264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.752946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.754915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.773641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.774277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.774574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.774825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.775637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.777284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.787843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 772/1580] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:04.788558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.798283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c50bf-d89d-4e3c-a396-21d70bbcc9cf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.803170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6870bd-7033-40bb-bc1a-ce828705445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.803804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6870bd-7033-40bb-bc1a-ce828705445f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.804083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6870bd-7033-40bb-bc1a-ce828705445f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.804357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6870bd-7033-40bb-bc1a-ce828705445f admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.805001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6870bd-7033-40bb-bc1a-ce828705445f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.806865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6870bd-7033-40bb-bc1a-ce828705445f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.839345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 809/1581] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.839345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.847962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8778ed-b5c2-4c89-911e-a7c81a14fb25 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.852637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.853232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.853521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.858064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.858064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.858064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.907731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 773/1582] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.907731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d8d8-f2ca-4630-8a17-b0c92706b9cf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.907731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d8d8-f2ca-4630-8a17-b0c92706b9cf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.907731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d8d8-f2ca-4630-8a17-b0c92706b9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.927096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 810/1583] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:04.927096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.938319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069937fd-ff02-4cda-91eb-351160248b7f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.939159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0bd0d8d8-f2ca-4630-8a17-b0c92706b9cf None None] MFA Rules not processed for user `670637f19e32493181e52e9df6c2f576`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:04.940424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0bd0d8d8-f2ca-4630-8a17-b0c92706b9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:04.943238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.943881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.944174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.944725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.945882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.947585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:04.958614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 811/1584] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:04.959408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:04.969073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d869-fb59-4f18-aee7-4db37a317681 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:04.974448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:04.975048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:04.975336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:04.975647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:04.976330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:04.978300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.008703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 774/1585] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:05.009552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0d8d8-f2ca-4630-8a17-b0c92706b9cf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.021769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0d8d8-f2ca-4630-8a17-b0c92706b9cf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.026347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.026938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.027222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.027496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.028209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.029843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.039744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 775/1586] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.040495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.050132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bed7b-2a1b-43b7-a27d-e468249fa747 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.054821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.055456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.055781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.056063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] PATH_INFO: `/v3/projects/606b2e9ca91341fe9bbe157935c5736e/users/f030155981d741f1a42b9b84ca5d6af4/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.071149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=606b2e9ca91341fe9bbe157935c5736e, user_id=f030155981d741f1a42b9b84ca5d6af4, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.075050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 08:00:05.075050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 08:00:05.075050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 08:00:05.075050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.076783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 812/1587] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:04 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:05.078262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.089778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a032fbe-2cc9-40cb-a1cc-4441ae43631d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.094624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.095220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.095504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.103897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.103897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.103897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.107980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 813/1588] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.109094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.118283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89960b41-5bba-477e-9ede-ebcd3d9b7fee admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.122828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.126853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.126853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.126853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.126853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.126853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.137334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 814/1589] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.138095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.148040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900952b6-89a1-47de-8886-ac2eb311c711 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.149064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 776/1590] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:05 2022] PUT /identity/v3/projects/606b2e9ca91341fe9bbe157935c5736e/users/f030155981d741f1a42b9b84ca5d6af4/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:05.149965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.208477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.209030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.209360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.209589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.210445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.212004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.214106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875366b-1059-4432-afff-4395f5da6291 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.219294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.219943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.220224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.220497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.221221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.221941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 815/1591] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.222729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.224608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.226506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:05.232208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c511ec-c1cc-4b8c-9fe5-4d3cd7e554e2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.235942 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 777/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.236889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.239855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b46cd-d3f4-4d57-93b6-5aad1ac66c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.240422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b46cd-d3f4-4d57-93b6-5aad1ac66c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.240694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b46cd-d3f4-4d57-93b6-5aad1ac66c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.240978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b46cd-d3f4-4d57-93b6-5aad1ac66c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.241655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b46cd-d3f4-4d57-93b6-5aad1ac66c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.243578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b46cd-d3f4-4d57-93b6-5aad1ac66c80 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.248500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317244f8-aad8-42bf-a1fb-56fa377c0713 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.254542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.255147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.255440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.255764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.256460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.258049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.268949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 778/1593] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.269764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.279070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9e551-28f3-4f70-8968-7b5b26cf186a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.284447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.285057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.285343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.285624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.286293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.287909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.300059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 779/1594] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.300845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.310133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34e243-e987-4307-adee-f01fee0be0f2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.315374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.322315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.322315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.322315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.322315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.322315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.328706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 780/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.329472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.338721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53219-c411-4c55-b70a-c1789a151be0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.343393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.344026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.344313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.344591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.345258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.346805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.355648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 816/1596] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:05 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:05.355648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3217-9e5a-4d6d-bbf9-746cc3bcf4fd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.355648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3217-9e5a-4d6d-bbf9-746cc3bcf4fd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.355648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3217-9e5a-4d6d-bbf9-746cc3bcf4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.361914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 781/1597] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.361914 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.368025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc8851-dae2-44ac-bb58-c6bcc29febfc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.375086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-98fc3217-9e5a-4d6d-bbf9-746cc3bcf4fd None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:05.376342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-98fc3217-9e5a-4d6d-bbf9-746cc3bcf4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:05.397080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:05.425996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.426626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.426917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.427199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.427929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.429544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.436290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 817/1598] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:05.437684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc3217-9e5a-4d6d-bbf9-746cc3bcf4fd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.440815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 782/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.442350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.447667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc3217-9e5a-4d6d-bbf9-746cc3bcf4fd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.451979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff8281-96ad-42dd-8a5a-582e4af1e36f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.454859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.455594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.455953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.456264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] PATH_INFO: `/v3/projects/7054fc89ba16433bb0669bc6a99044a5/users/0d2c7d95047445ee9d44640b5372e0e5/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.456777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.457443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.457732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.458037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] PATH_INFO: `/v3/projects/9d14e0b976344ea7a152250846d587dd/users/7280265a21ab4cfda4538a3064ab8853/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.472317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7054fc89ba16433bb0669bc6a99044a5, user_id=0d2c7d95047445ee9d44640b5372e0e5, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.473324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d14e0b976344ea7a152250846d587dd, user_id=7280265a21ab4cfda4538a3064ab8853, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.474960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.475990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.550507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 818/1600] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:05 2022] PUT /identity/v3/projects/7054fc89ba16433bb0669bc6a99044a5/users/0d2c7d95047445ee9d44640b5372e0e5/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:05.551385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.557072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 783/1601] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:05 2022] PUT /identity/v3/projects/9d14e0b976344ea7a152250846d587dd/users/7280265a21ab4cfda4538a3064ab8853/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:05.558015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.617395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14871ad5-7ca0-41a6-85ab-8829a4638401 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.621011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689705d9-cf29-4c5c-9148-645d4a952161 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.622057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.622750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.622966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.623247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.624607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.626221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.626337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.626885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.627167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.627443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] PATH_INFO: `/v3/projects/82a5351f1ca84a81b22ee93f1dfeb76a/users/e497e3e62c134aba989f961ef5b54936/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.636011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 819/1602] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:05.636791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.642430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a5351f1ca84a81b22ee93f1dfeb76a, user_id=e497e3e62c134aba989f961ef5b54936, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.645115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.648351 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a98098-84a2-4d5d-96fa-a2dcff583e4b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.653199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.660557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.660557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.660557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.660557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.660557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.721387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 784/1603] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:05 2022] PUT /identity/v3/projects/82a5351f1ca84a81b22ee93f1dfeb76a/users/e497e3e62c134aba989f961ef5b54936/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:05.722239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.731008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:05.755998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 820/1604] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:05 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:05.756849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.791580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdf587-faa5-4f50-9285-5b30d4206ae6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.797423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e357d-7d6b-4338-979f-37c852616c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.798088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e357d-7d6b-4338-979f-37c852616c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.798373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e357d-7d6b-4338-979f-37c852616c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.798654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e357d-7d6b-4338-979f-37c852616c4a admin admin] PATH_INFO: `/v3/projects/768f11935fcd4080b5045613f6b7cb28/users/570de5da908c4b31a8a22706642aafb4/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.813918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e357d-7d6b-4338-979f-37c852616c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=768f11935fcd4080b5045613f6b7cb28, user_id=570de5da908c4b31a8a22706642aafb4, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.814129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663b04b-651f-4439-b758-6e85623c26c0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.816694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e357d-7d6b-4338-979f-37c852616c4a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.847666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:05.863151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.863951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.864238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.864516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.873411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.874979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.884385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 821/1605] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:05.885242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:05.894128 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5424f-8c4b-41d2-985f-36392fc29b57 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:05.899954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:05.900666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.900967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.901264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] PATH_INFO: `/v3/projects/d98ebfe64a6a4a5e8aacbfc607a619a9/users/cfaa27acb8ae4710b883e504e0f59d74/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.904293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 785/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:05 2022] PUT /identity/v3/projects/768f11935fcd4080b5045613f6b7cb28/users/570de5da908c4b31a8a22706642aafb4/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:05.905873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1847e4a-b3f8-4049-ac4a-0243449ec02b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.906166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1847e4a-b3f8-4049-ac4a-0243449ec02b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.906468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1847e4a-b3f8-4049-ac4a-0243449ec02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:05.919080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d98ebfe64a6a4a5e8aacbfc607a619a9, user_id=cfaa27acb8ae4710b883e504e0f59d74, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:05.922695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:05.927958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b1847e4a-b3f8-4049-ac4a-0243449ec02b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:05.929163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b1847e4a-b3f8-4049-ac4a-0243449ec02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:05.986394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 786/1607] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:05.987571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49c26d-9d48-44c6-b7ae-4bcadbf4717a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:05.987859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49c26d-9d48-44c6-b7ae-4bcadbf4717a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:05.988187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49c26d-9d48-44c6-b7ae-4bcadbf4717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.016953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4d49c26d-9d48-44c6-b7ae-4bcadbf4717a None None] MFA Rules not processed for user `7280265a21ab4cfda4538a3064ab8853`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:06.018223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4d49c26d-9d48-44c6-b7ae-4bcadbf4717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:06.020632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 822/1608] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:05 2022] PUT /identity/v3/projects/d98ebfe64a6a4a5e8aacbfc607a619a9/users/cfaa27acb8ae4710b883e504e0f59d74/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:06.031924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.096850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 787/1609] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:06.096850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59cc76-54ec-447a-accd-cb2409e7469c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.096850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59cc76-54ec-447a-accd-cb2409e7469c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.096850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59cc76-54ec-447a-accd-cb2409e7469c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.109600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8a8de-20de-40bb-b0c1-2667a07044f6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.114501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381901e4-e46c-47d5-a034-cec6f06f0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.115180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381901e4-e46c-47d5-a034-cec6f06f0702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.115474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381901e4-e46c-47d5-a034-cec6f06f0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.115792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381901e4-e46c-47d5-a034-cec6f06f0702 admin admin] PATH_INFO: `/v3/projects/aae673506a1a4ea2b3209c8a86b3a33b/users/972bc4e9bc9e49b4a3bc2a693b05e4ab/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.123857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5d59cc76-54ec-447a-accd-cb2409e7469c None None] MFA Rules not processed for user `e497e3e62c134aba989f961ef5b54936`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:06.125067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5d59cc76-54ec-447a-accd-cb2409e7469c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:06.131201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381901e4-e46c-47d5-a034-cec6f06f0702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae673506a1a4ea2b3209c8a86b3a33b, user_id=972bc4e9bc9e49b4a3bc2a693b05e4ab, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.133267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:06.133948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381901e4-e46c-47d5-a034-cec6f06f0702 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.190247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 788/1610] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:06.191111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59cc76-54ec-447a-accd-cb2409e7469c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.201607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59cc76-54ec-447a-accd-cb2409e7469c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.208863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.209491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.209784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.210065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.210764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.212411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.217084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 823/1611] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:06 2022] PUT /identity/v3/projects/aae673506a1a4ea2b3209c8a86b3a33b/users/972bc4e9bc9e49b4a3bc2a693b05e4ab/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:06.217084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bc33a-d523-4b29-be22-8caa0753fd83 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.217084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bc33a-d523-4b29-be22-8caa0753fd83 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.217084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bc33a-d523-4b29-be22-8caa0753fd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.225040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 789/1612] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:06.225892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.227769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:06.241133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bb5bc33a-d523-4b29-be22-8caa0753fd83 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:06.242401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bb5bc33a-d523-4b29-be22-8caa0753fd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:06.305738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aee50-2871-4710-9c32-97ce86265d8b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.315605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 824/1613] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:06.317080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bfe80-6576-4dfd-a9f8-81194787b8b6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.317375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bfe80-6576-4dfd-a9f8-81194787b8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.317655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bfe80-6576-4dfd-a9f8-81194787b8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.346444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e17bfe80-6576-4dfd-a9f8-81194787b8b6 None None] MFA Rules not processed for user `cfaa27acb8ae4710b883e504e0f59d74`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:06.348854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e17bfe80-6576-4dfd-a9f8-81194787b8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:06.359649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.360322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.360612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.361022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.373224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.375088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.385466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:06.386599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 790/1614] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:06.391070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.427452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 825/1615] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:06.428433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bfe80-6576-4dfd-a9f8-81194787b8b6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.440607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bfe80-6576-4dfd-a9f8-81194787b8b6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.445968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.446600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.446890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.447759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.448945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.452335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.461214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85541ad3-1fa8-4bac-b914-7d92d938c0dc service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.466439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caf0f3-21cb-4cc5-b9f9-9bcfe5e8a019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.467291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf0f3-21cb-4cc5-b9f9-9bcfe5e8a019 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.467641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf0f3-21cb-4cc5-b9f9-9bcfe5e8a019 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.467928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf0f3-21cb-4cc5-b9f9-9bcfe5e8a019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.477995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:06.535566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caf0f3-21cb-4cc5-b9f9-9bcfe5e8a019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.536857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caf0f3-21cb-4cc5-b9f9-9bcfe5e8a019 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.546347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 791/1616] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:06.547847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df10ab-71a3-49b9-9881-31e823df4570 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.547847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df10ab-71a3-49b9-9881-31e823df4570 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.548071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df10ab-71a3-49b9-9881-31e823df4570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.569715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-38df10ab-71a3-49b9-9881-31e823df4570 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:06.571267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-38df10ab-71a3-49b9-9881-31e823df4570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:06.585978 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 792/1617] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:06.586932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df10ab-71a3-49b9-9881-31e823df4570 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.588810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 826/1618] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:06 2022] POST /identity/v3/projects => generated 389 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:06.589826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.602754 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df10ab-71a3-49b9-9881-31e823df4570 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.648663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094306c3-1175-482d-8fcd-a83ea7e974e7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.652700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.653351 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.653633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.653904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.662855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.663578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.663872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.664152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] PATH_INFO: `/v3/projects/ea946ed607ca43ffbb30f1622fc5b0b8/users/2ccbed04bb1b479dbb1aad7c8f59b165/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.682084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea946ed607ca43ffbb30f1622fc5b0b8, user_id=2ccbed04bb1b479dbb1aad7c8f59b165, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.685907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.771282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.771615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 793/1619] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:06 2022] PUT /identity/v3/projects/ea946ed607ca43ffbb30f1622fc5b0b8/users/2ccbed04bb1b479dbb1aad7c8f59b165/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:06.772641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.773806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.796448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 827/1620] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:06.799093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:06.823703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d5532-351c-42b9-ac1a-4577eb68fcfa service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.909825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14f45e-9b2c-4d66-9036-920d3f66744e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:06.915264 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.927026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.927026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.927026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.927026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.927026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:06.927591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:06.932092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:06.932740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:06.933163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:06.933444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:06.934127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:06.936995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.036205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 828/1621] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:06 2022] POST /identity/v3/users => generated 301 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:07.037805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.041296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 794/1622] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:06 2022] POST /identity/v3/projects => generated 373 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:07.042713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.060935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:07.150694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a3784-e7b7-4c72-9146-312bd25e2b72 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.195135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b88c5-c244-4493-a09a-d0c12e83f6e1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.202738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.203685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.204138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.204829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.205172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.207273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.220833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 795/1623] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:07.223624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.224725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.225828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.225828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.226048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.234423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747480e-4241-4d6e-a5a7-7099d1d655c4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.237622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.239216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.240731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.240886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.241149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.241433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.242263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.244297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.251127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 829/1624] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:07.252058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.258567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 796/1625] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:07.260828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.269885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865f581-1425-4ee0-9023-a7e66397e69d service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.275841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a90ca-16a5-4771-8a7e-f475777a432a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.281577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ca24d-68fd-4b36-91b5-71ca06a60945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.282225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ca24d-68fd-4b36-91b5-71ca06a60945 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.282484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ca24d-68fd-4b36-91b5-71ca06a60945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.282753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ca24d-68fd-4b36-91b5-71ca06a60945 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.283420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ca24d-68fd-4b36-91b5-71ca06a60945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.285428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.285582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ca24d-68fd-4b36-91b5-71ca06a60945 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.285678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.285986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.286282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] PATH_INFO: `/v3/projects/ea946ed607ca43ffbb30f1622fc5b0b8/users/2ccbed04bb1b479dbb1aad7c8f59b165/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.288637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea946ed607ca43ffbb30f1622fc5b0b8, user_id=2ccbed04bb1b479dbb1aad7c8f59b165, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.291350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.307206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 797/1626] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:07 2022] PUT /identity/v3/projects/ea946ed607ca43ffbb30f1622fc5b0b8/users/2ccbed04bb1b479dbb1aad7c8f59b165/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:07.316865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.377309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 830/1627] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:07 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:07.377309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829e1df-f826-49e1-addc-6c793c6aa161 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.377309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829e1df-f826-49e1-addc-6c793c6aa161 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.377309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829e1df-f826-49e1-addc-6c793c6aa161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.385662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd711d49-7341-4b61-bec7-a128988a7411 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.392456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.393240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.393597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.393946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] PATH_INFO: `/v3/projects/10aaebd207d44c4c8bc2fe8dbb49d20d/users/3d9253d916d34cad96d31d847df8e888/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.420182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10aaebd207d44c4c8bc2fe8dbb49d20d, user_id=3d9253d916d34cad96d31d847df8e888, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.423498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.435006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8829e1df-f826-49e1-addc-6c793c6aa161 None None] MFA Rules not processed for user `2ccbed04bb1b479dbb1aad7c8f59b165`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:07.436385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8829e1df-f826-49e1-addc-6c793c6aa161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:07.522287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 798/1628] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:07 2022] PUT /identity/v3/projects/10aaebd207d44c4c8bc2fe8dbb49d20d/users/3d9253d916d34cad96d31d847df8e888/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:07.522287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.593852 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 831/1629] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:07.595193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829e1df-f826-49e1-addc-6c793c6aa161 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.617164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a338063-2432-4bd4-8fc2-b29884ed2025 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.629322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.630406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.631028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.631772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.656317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.657152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829e1df-f826-49e1-addc-6c793c6aa161 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.657247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.657324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.657777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.658065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.658337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.659081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.667439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.682416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 799/1630] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:07.687670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.690083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 832/1631] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:07.691978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.701229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:07.706145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:07.720658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94502f22-3e2b-461a-87cf-dc1d8f475f95 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.734230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.734937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.735219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.735500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] PATH_INFO: `/v3/projects/fc2331557ef44708b92e1945be7fa2a6/users/d033355ee83d4dea82a2e9b7a9652b5b/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.767703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2331557ef44708b92e1945be7fa2a6, user_id=d033355ee83d4dea82a2e9b7a9652b5b, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.777430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.800840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2f99b-b514-469f-8d73-24f1af4da32d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:07.804800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:07.805444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:07.805899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:07.806024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:07.878022 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 833/1632] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:07 2022] PUT /identity/v3/projects/fc2331557ef44708b92e1945be7fa2a6/users/d033355ee83d4dea82a2e9b7a9652b5b/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:07.879359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.932746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:07.934411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:07.961626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 800/1633] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:07.962683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:07.990622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d070aa-f275-42a4-91bf-8f225ae568da service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:08.025974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d04d0a-2baf-428f-aa72-880e6673cbba admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:08.039713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5e640-2d9e-49b1-92a2-f448104b3600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:08.040413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5e640-2d9e-49b1-92a2-f448104b3600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.041063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5e640-2d9e-49b1-92a2-f448104b3600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.041356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5e640-2d9e-49b1-92a2-f448104b3600 admin admin] PATH_INFO: `/v3/projects/10aaebd207d44c4c8bc2fe8dbb49d20d/users/3d9253d916d34cad96d31d847df8e888/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.047246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5e640-2d9e-49b1-92a2-f448104b3600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10aaebd207d44c4c8bc2fe8dbb49d20d, user_id=3d9253d916d34cad96d31d847df8e888, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:08.054453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5e640-2d9e-49b1-92a2-f448104b3600 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:08.090711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 834/1634] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:07 2022] PUT /identity/v3/projects/10aaebd207d44c4c8bc2fe8dbb49d20d/users/3d9253d916d34cad96d31d847df8e888/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:08.092600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8be3dc-8c7e-45ec-a42a-b62a4270b670 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.093063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8be3dc-8c7e-45ec-a42a-b62a4270b670 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.093352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8be3dc-8c7e-45ec-a42a-b62a4270b670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.125714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:08.126974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.127417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.128329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.129203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:08.132129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:08.147856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8a8be3dc-8c7e-45ec-a42a-b62a4270b670 None None] MFA Rules not processed for user `3d9253d916d34cad96d31d847df8e888`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:08.155820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8a8be3dc-8c7e-45ec-a42a-b62a4270b670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:08.158320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 801/1635] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:08.159282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:08.190652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4060e1-3e79-4a3e-b84f-b43bec7295df admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:08.273146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd24a19-6e91-42be-98ea-47ae6447f831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:08.273977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd24a19-6e91-42be-98ea-47ae6447f831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.274331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd24a19-6e91-42be-98ea-47ae6447f831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.274733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd24a19-6e91-42be-98ea-47ae6447f831 admin admin] PATH_INFO: `/v3/projects/fc2331557ef44708b92e1945be7fa2a6/users/d033355ee83d4dea82a2e9b7a9652b5b/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.277091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd24a19-6e91-42be-98ea-47ae6447f831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2331557ef44708b92e1945be7fa2a6, user_id=d033355ee83d4dea82a2e9b7a9652b5b, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:08.279973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd24a19-6e91-42be-98ea-47ae6447f831 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:08.290998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 835/1636] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:08.292029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 802/1637] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:08 2022] PUT /identity/v3/projects/fc2331557ef44708b92e1945be7fa2a6/users/d033355ee83d4dea82a2e9b7a9652b5b/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:08.292155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8be3dc-8c7e-45ec-a42a-b62a4270b670 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:08.303062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b757883-852a-42b1-ae0d-6aefe67686e2 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.303356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b757883-852a-42b1-ae0d-6aefe67686e2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.303943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b757883-852a-42b1-ae0d-6aefe67686e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.353656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7b757883-852a-42b1-ae0d-6aefe67686e2 None None] MFA Rules not processed for user `d033355ee83d4dea82a2e9b7a9652b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:08.354502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7b757883-852a-42b1-ae0d-6aefe67686e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:08.406552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8be3dc-8c7e-45ec-a42a-b62a4270b670 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:08.524214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 803/1638] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:08.524214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b757883-852a-42b1-ae0d-6aefe67686e2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:08.524544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:08.524544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.524544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.524544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.527397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:08.564596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b757883-852a-42b1-ae0d-6aefe67686e2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:08.571301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:08.573158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.573764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.575033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.630929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:08.633170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:08.650377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 836/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:08.678046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:08.680133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:08.727144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 804/1640] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:08.732330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:08.749926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fb794-cfd0-4743-9398-51f0f6085ffa service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:08.756538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:08.766023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301769d-1efd-4c5d-bfc4-fafa17c91a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:08.766799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301769d-1efd-4c5d-bfc4-fafa17c91a30 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:08.767251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301769d-1efd-4c5d-bfc4-fafa17c91a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:08.767519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301769d-1efd-4c5d-bfc4-fafa17c91a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:08.788762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301769d-1efd-4c5d-bfc4-fafa17c91a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:08.790425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301769d-1efd-4c5d-bfc4-fafa17c91a30 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:08.851502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 837/1641] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:09.478015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:09.614750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e67da3-9667-4bf0-ada2-1d688ed79a9b service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:09.622149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:09.623248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:09.624498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:09.624997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] PATH_INFO: `/v3/users/e497e3e62c134aba989f961ef5b54936` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:09.626457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e497e3e62c134aba989f961ef5b54936)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:09.628174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 08:00:09.628174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: warnings.warn(msg) Aug 01 08:00:09.628174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: Aug 01 08:00:09.631275 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:09.653640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:10.025702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e497e3e62c134aba989f961ef5b54936'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:10.039788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e497e3e62c134aba989f961ef5b54936'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:10.139135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e497e3e62c134aba989f961ef5b54936'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:10.140153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e497e3e62c134aba989f961ef5b54936'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:10.169970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e497e3e62c134aba989f961ef5b54936'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:10.185409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 805/1642] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:09 2022] DELETE /identity/v3/users/e497e3e62c134aba989f961ef5b54936 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:10.187139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:10.188016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:10.188412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:10.246413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:10.251899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:10.388299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 838/1643] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:10.449415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:10.451455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:10.646745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ff5a-78f0-47b5-b293-abbc7b4e902c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:10.652239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc595f5-1397-4d15-ac6b-d9d76e34977a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:10.652847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc595f5-1397-4d15-ac6b-d9d76e34977a service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:10.653252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc595f5-1397-4d15-ac6b-d9d76e34977a service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:10.655400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc595f5-1397-4d15-ac6b-d9d76e34977a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:10.675898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc595f5-1397-4d15-ac6b-d9d76e34977a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:10.678222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc595f5-1397-4d15-ac6b-d9d76e34977a service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:10.693762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 806/1644] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:10.746450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:10.747990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:10.768475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f619d6-27ea-468c-b639-dc5e1c8cdd0f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:10.783073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:10.783779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:10.784062 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:10.784338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] PATH_INFO: `/v3/users/cfaa27acb8ae4710b883e504e0f59d74` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:10.785538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfaa27acb8ae4710b883e504e0f59d74)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:10.792263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:10.855724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0a91a-a9b6-4b73-8516-f4f987f7ae02 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:10.857727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0a91a-a9b6-4b73-8516-f4f987f7ae02 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:10.857727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0a91a-a9b6-4b73-8516-f4f987f7ae02 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:10.859518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 807/1645] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 08:00:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:10.872826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93722767-3cd3-47f7-ba1b-da10d8eab67a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:10.873483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93722767-3cd3-47f7-ba1b-da10d8eab67a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:10.873873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93722767-3cd3-47f7-ba1b-da10d8eab67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:10.978405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-93722767-3cd3-47f7-ba1b-da10d8eab67a None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:10.980436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-93722767-3cd3-47f7-ba1b-da10d8eab67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:11.043429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:11.054895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfaa27acb8ae4710b883e504e0f59d74'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:11.146265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 808/1646] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 08:00:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:11.151834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d5e2f-b6f4-4151-9f25-62da5ff46071 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:11.152201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d5e2f-b6f4-4151-9f25-62da5ff46071 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:11.152631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d5e2f-b6f4-4151-9f25-62da5ff46071 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:11.155047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 809/1647] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 08:00:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:11.161750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:11.162226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:11.163270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:11.190378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfaa27acb8ae4710b883e504e0f59d74'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:11.208529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfaa27acb8ae4710b883e504e0f59d74'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:11.210701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfaa27acb8ae4710b883e504e0f59d74'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:11.214205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-051e3ec5-273a-4375-9296-f4360a08719b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfaa27acb8ae4710b883e504e0f59d74'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:11.216912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:11.221271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:11.235797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 810/1648] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:00:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:11.254286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:11.254572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 839/1649] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:10 2022] DELETE /identity/v3/users/cfaa27acb8ae4710b883e504e0f59d74 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:11.256229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:11.256745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:11.257232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:11.259443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:11.278613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:11.280898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:11.282600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:11.407309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e5db3-37ec-48cb-90e6-62538fe6504f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:11.415419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 840/1650] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:11.429593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:11.431147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:11.518354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:11.519295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:11.519725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:11.520076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:11.563785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44de41-a164-46d4-b695-f0822b08f97e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:11.576081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:11.578074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:11.579570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:11.580098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:11.616288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:11.620814 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:11.626238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:11.628220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:11.643549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 811/1651] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:11.676591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 841/1652] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:12.459077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:12.461579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:12.492977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99650605-fe59-4e4d-a056-743c1f647153 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:12.502276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:12.504702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:12.504702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:12.504702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] PATH_INFO: `/v3/projects/82a5351f1ca84a81b22ee93f1dfeb76a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:12.505900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a5351f1ca84a81b22ee93f1dfeb76a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:12.508333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:12.687603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a5351f1ca84a81b22ee93f1dfeb76a', 'request_id': 'req-0ab70f49-f600-41ba-9281-bd7d75f94d85'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:12.697563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a5351f1ca84a81b22ee93f1dfeb76a', 'request_id': 'req-0ab70f49-f600-41ba-9281-bd7d75f94d85'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:12.716266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a5351f1ca84a81b22ee93f1dfeb76a', 'request_id': 'req-0ab70f49-f600-41ba-9281-bd7d75f94d85'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:12.832964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] The token cache is being invalidate because project 82a5351f1ca84a81b22ee93f1dfeb76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:12.833672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:12.834709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:12.836492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 812/1653] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:12 2022] DELETE /identity/v3/projects/82a5351f1ca84a81b22ee93f1dfeb76a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:12.840737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:12.865031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1529343-8f51-436c-ad31-988946d65639 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:13.055766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:13.055970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:13.064896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:13.064896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] PATH_INFO: `/v3/users/3d9253d916d34cad96d31d847df8e888` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:13.064896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9253d916d34cad96d31d847df8e888)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:13.066622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:13.069056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:13.071424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:13.084253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:13.105921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab70f49-f600-41ba-9281-bd7d75f94d85 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:13.126754 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:13.127765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:13.128225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:13.128671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] PATH_INFO: `/v3/projects/d98ebfe64a6a4a5e8aacbfc607a619a9` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:13.132302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d98ebfe64a6a4a5e8aacbfc607a619a9)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:13.134650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:13.405687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd98ebfe64a6a4a5e8aacbfc607a619a9', 'request_id': 'req-a267dc83-6456-42bd-8475-cdf7299f7f89'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.413713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9253d916d34cad96d31d847df8e888'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.423602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd98ebfe64a6a4a5e8aacbfc607a619a9', 'request_id': 'req-a267dc83-6456-42bd-8475-cdf7299f7f89'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.441138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd98ebfe64a6a4a5e8aacbfc607a619a9', 'request_id': 'req-a267dc83-6456-42bd-8475-cdf7299f7f89'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.556875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9253d916d34cad96d31d847df8e888'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.565439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9253d916d34cad96d31d847df8e888'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.576075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9253d916d34cad96d31d847df8e888'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.576954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9253d916d34cad96d31d847df8e888'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.613232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 842/1654] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:12 2022] DELETE /identity/v3/users/3d9253d916d34cad96d31d847df8e888 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:13.670848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] The token cache is being invalidate because project d98ebfe64a6a4a5e8aacbfc607a619a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:13.671428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.673079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:13.679081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 813/1655] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:13 2022] DELETE /identity/v3/projects/d98ebfe64a6a4a5e8aacbfc607a619a9 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:13.683221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:13.684731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:13.824629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6637a-deaf-499d-99d8-7a11db836cca admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:13.841717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:13.843134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:13.843433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:13.843749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] PATH_INFO: `/v3/users/d033355ee83d4dea82a2e9b7a9652b5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:13.848468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d033355ee83d4dea82a2e9b7a9652b5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:13.850787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:14.130275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd033355ee83d4dea82a2e9b7a9652b5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:14.133722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:14.224570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd033355ee83d4dea82a2e9b7a9652b5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:14.235311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd033355ee83d4dea82a2e9b7a9652b5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:14.240738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd033355ee83d4dea82a2e9b7a9652b5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:14.241801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bf622f82-6a40-4832-bbef-6eba287b671c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd033355ee83d4dea82a2e9b7a9652b5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:14.315316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 843/1656] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:13 2022] DELETE /identity/v3/users/d033355ee83d4dea82a2e9b7a9652b5b => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:14.430792 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:14.432932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:14.638288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267dc83-6456-42bd-8475-cdf7299f7f89 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:14.772725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:14.775021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:14.776350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:14.776948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] PATH_INFO: `/v3/projects/10aaebd207d44c4c8bc2fe8dbb49d20d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:14.781713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=10aaebd207d44c4c8bc2fe8dbb49d20d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:14.784782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:14.986192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10aaebd207d44c4c8bc2fe8dbb49d20d', 'request_id': 'req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:14.998999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10aaebd207d44c4c8bc2fe8dbb49d20d', 'request_id': 'req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:15.008332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10aaebd207d44c4c8bc2fe8dbb49d20d', 'request_id': 'req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:15.237982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] The token cache is being invalidate because project 10aaebd207d44c4c8bc2fe8dbb49d20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:15.268638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:15.268638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1fe3b219-9bb7-4937-80a4-c34e0e78ef1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:15.268638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 814/1657] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:14 2022] DELETE /identity/v3/projects/10aaebd207d44c4c8bc2fe8dbb49d20d => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:15.269738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f4e8-09c3-4fe9-96da-5d1d381c658d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:15.269738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f4e8-09c3-4fe9-96da-5d1d381c658d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:15.269738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f4e8-09c3-4fe9-96da-5d1d381c658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:15.340677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ecd9f4e8-09c3-4fe9-96da-5d1d381c658d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:15.340677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ecd9f4e8-09c3-4fe9-96da-5d1d381c658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:15.419847 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:15.426296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 844/1658] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:15.435191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f262d5-2d9c-4a28-9e67-3b261e36c022 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:15.437559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f262d5-2d9c-4a28-9e67-3b261e36c022 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:15.437883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f262d5-2d9c-4a28-9e67-3b261e36c022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:15.466446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-34f262d5-2d9c-4a28-9e67-3b261e36c022 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:15.468440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34f262d5-2d9c-4a28-9e67-3b261e36c022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:15.479221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 815/1659] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:15.485443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f4e8-09c3-4fe9-96da-5d1d381c658d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:15.497775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f4e8-09c3-4fe9-96da-5d1d381c658d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:15.509087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:15.510242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:15.510683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:15.511035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:15.512006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:15.512390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:15.515628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:15.533724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 845/1660] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:15.538677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f262d5-2d9c-4a28-9e67-3b261e36c022 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:15.553252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f262d5-2d9c-4a28-9e67-3b261e36c022 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:15.557746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:15.566325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:15.568549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:15.569074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:15.569429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:15.570275 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:15.573925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:15.710024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 816/1661] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:15 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:15.714799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:15.729032 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac27c29-caec-4c58-975e-e0e68a0a255e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:15.768216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:15.772118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:15.886968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca375-0024-4117-af20-8d72291baee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:15.887656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca375-0024-4117-af20-8d72291baee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:15.887952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca375-0024-4117-af20-8d72291baee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:15.888393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca375-0024-4117-af20-8d72291baee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:15.889215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ca375-0024-4117-af20-8d72291baee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:15.895504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ca375-0024-4117-af20-8d72291baee1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:15.917601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f40eed-5a6c-4712-b7f1-9c621bb578be admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:15.963715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:15.964586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:15.968254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:15.968563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] PATH_INFO: `/v3/projects/fc2331557ef44708b92e1945be7fa2a6` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:15.972361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2331557ef44708b92e1945be7fa2a6)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:15.978870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:16.028461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 846/1662] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:15 2022] POST /identity/v3/users => generated 292 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:16.028461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f10d7-7323-493b-ab89-dcae9c8c372a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.028461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f10d7-7323-493b-ab89-dcae9c8c372a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.028780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f10d7-7323-493b-ab89-dcae9c8c372a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.030159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 847/1663] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 08:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:16.034640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f10d7-7323-493b-ab89-dcae9c8c372a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:16.075516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f10d7-7323-493b-ab89-dcae9c8c372a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:16.085948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a6803-07f5-48bb-b1f0-1db662ea8728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:16.088807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a6803-07f5-48bb-b1f0-1db662ea8728 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.089168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a6803-07f5-48bb-b1f0-1db662ea8728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.092450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a6803-07f5-48bb-b1f0-1db662ea8728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.092450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a6803-07f5-48bb-b1f0-1db662ea8728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:16.094617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a6803-07f5-48bb-b1f0-1db662ea8728 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:16.129505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 848/1664] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:16.130940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228dbb6-5b1f-44f4-9b01-fce126eef202 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.131235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228dbb6-5b1f-44f4-9b01-fce126eef202 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.131751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228dbb6-5b1f-44f4-9b01-fce126eef202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.189014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0228dbb6-5b1f-44f4-9b01-fce126eef202 None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:16.190596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0228dbb6-5b1f-44f4-9b01-fce126eef202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:16.270588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2331557ef44708b92e1945be7fa2a6', 'request_id': 'req-cb0c3aa6-e788-40aa-a76f-d6e94601c927'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:16.278244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2331557ef44708b92e1945be7fa2a6', 'request_id': 'req-cb0c3aa6-e788-40aa-a76f-d6e94601c927'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:16.286692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2331557ef44708b92e1945be7fa2a6', 'request_id': 'req-cb0c3aa6-e788-40aa-a76f-d6e94601c927'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:16.402606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] The token cache is being invalidate because project fc2331557ef44708b92e1945be7fa2a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:16.402989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:16.405680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:16.413799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 817/1665] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:15 2022] DELETE /identity/v3/projects/fc2331557ef44708b92e1945be7fa2a6 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:16.414543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:16.417697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:16.429849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 849/1666] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 08:00:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:16.431850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e6ce2-29b0-49b0-bbc0-35ca46cc803d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.434571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e6ce2-29b0-49b0-bbc0-35ca46cc803d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.436698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e6ce2-29b0-49b0-bbc0-35ca46cc803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.481871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f01e6ce2-29b0-49b0-bbc0-35ca46cc803d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:16.483943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f01e6ce2-29b0-49b0-bbc0-35ca46cc803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:16.588696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c3aa6-e788-40aa-a76f-d6e94601c927 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:16.603516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:16.608813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.609121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.609409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.610177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:16.611878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:16.648936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 850/1668] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:16.658156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921825-8fb5-49b3-b667-458f9669e445 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.658156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921825-8fb5-49b3-b667-458f9669e445 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.658156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921825-8fb5-49b3-b667-458f9669e445 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.658515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 818/1668] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:16.658870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:16.665186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 851/1669] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 08:00:16 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:16.666631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ba75e-35c6-494c-a1e7-73133f72a6de None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.666924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ba75e-35c6-494c-a1e7-73133f72a6de None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.667204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ba75e-35c6-494c-a1e7-73133f72a6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.681329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5a0b9-61b8-4030-8790-6c1b8fec0566 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:16.696837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17745fb-f9db-4aee-b6b7-aa4e03c40183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:16.697539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17745fb-f9db-4aee-b6b7-aa4e03c40183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.697826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17745fb-f9db-4aee-b6b7-aa4e03c40183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.698105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17745fb-f9db-4aee-b6b7-aa4e03c40183 admin admin] PATH_INFO: `/v3/projects/8b04028ada0b456c89dc71071a51f57e/users/80d2204c01aa4a40b26517d7b80d19b7/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.718370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-036ba75e-35c6-494c-a1e7-73133f72a6de None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:16.719740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-036ba75e-35c6-494c-a1e7-73133f72a6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:16.747414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17745fb-f9db-4aee-b6b7-aa4e03c40183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b04028ada0b456c89dc71071a51f57e, user_id=80d2204c01aa4a40b26517d7b80d19b7, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:16.747414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17745fb-f9db-4aee-b6b7-aa4e03c40183 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:16.852945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 819/1670] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:16 2022] PUT /identity/v3/projects/8b04028ada0b456c89dc71071a51f57e/users/80d2204c01aa4a40b26517d7b80d19b7/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:16.863089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.863089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.863089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.900843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 852/1671] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:00:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:16.909245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231a0a-48d8-45f7-9325-9a6fb923e678 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:16.913847 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231a0a-48d8-45f7-9325-9a6fb923e678 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:16.914167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231a0a-48d8-45f7-9325-9a6fb923e678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:16.928218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:16.929482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:16.991767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c6231a0a-48d8-45f7-9325-9a6fb923e678 None None] MFA Rules not processed for user `80d2204c01aa4a40b26517d7b80d19b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:16.993031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c6231a0a-48d8-45f7-9325-9a6fb923e678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:17.114042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 820/1672] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:17.114042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:17.115309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:17.151628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 853/1673] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:17.152883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6231a0a-48d8-45f7-9325-9a6fb923e678 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:17.174736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6231a0a-48d8-45f7-9325-9a6fb923e678 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:17.195685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:17.200446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:17.201872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:17.202163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:17.202953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:17.205074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:17.207258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:17.241192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 854/1674] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:17.241192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:17.250203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ea00e-56ee-4ec5-84e2-42c7601651e9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:17.256679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:17.257314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:17.257601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:17.257880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:17.258630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:17.268577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:17.284703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e49b-d66e-4576-b8dd-7c7382ac13ff None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:17.288872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:17.289525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:17.289811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:17.290088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:17.436243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 855/1675] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:17 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:17.443272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:17.495768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:17.500265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:17.530801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 821/1676] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:17.543834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:17.556932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:17.587765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50be5c0-122a-4357-8098-87252a4829d0 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:17.590126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:17.637125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57535-20cd-48ab-b264-b4a734ff64d6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:17.654882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:17.655639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:17.655929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:17.656210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:17.657403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:17.659548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:17.786177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:17.787454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:17.787805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:17.788135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:17.788840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:17.791613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:17.818242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 856/1677] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:17 2022] POST /identity/v3/projects => generated 379 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:17.822966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:17.921033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 822/1678] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:17 2022] POST /identity/v3/users => generated 298 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:17.925967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:17.952497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980051d-a714-465b-8ca0-11fc5f32626a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:17.962407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfad5a-c2dd-4f13-ab3f-1ea9df241e2a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:17.963352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:17.964182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:17.965080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:17.966157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:17.968243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:17.972154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:17.976319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:17.978209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:17.978377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:17.981008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:17.981811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:17.987906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:17.997350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 823/1679] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:18.002664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:18.022480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55214e8-6616-475e-b53f-260ad046333d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:18.028210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:18.029516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:18.029906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:18.030283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] PATH_INFO: `/v3/projects/77ebde9fdeef450f884905d8dff31900/users/d249a356ebd54bf3b5240f5b372bd61f/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:18.052216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ebde9fdeef450f884905d8dff31900, user_id=d249a356ebd54bf3b5240f5b372bd61f, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:18.056457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:18.111578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 857/1680] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:17 2022] POST /identity/v3/users => generated 290 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:18.115817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:18.141973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b34e9-e71b-4405-b8de-2d382e9f50b3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:18.163240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:18.163850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:18.164139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:18.164419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:18.165519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:18.167293 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:18.188364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 824/1681] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:18 2022] PUT /identity/v3/projects/77ebde9fdeef450f884905d8dff31900/users/d249a356ebd54bf3b5240f5b372bd61f/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:18.192584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:18.210259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 858/1682] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:18.212940 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:18.252127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369cd91-393a-4fd1-ac74-cb248ed77fcb admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:18.364429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28dbd8-880b-4307-ba48-c6d5ee38e609 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:18.380427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:18.381330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:18.383913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:18.384352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:18.387271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:18.390238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:18.422517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 825/1683] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:18.423888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:18.437451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521db19-769c-46a3-bad3-95640c0cef68 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:18.439105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:18.439901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:18.442549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:18.443115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:18.443899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:18.445172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c718b3-5bdb-4af4-9681-716c340b0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:18.447436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c718b3-5bdb-4af4-9681-716c340b0057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:18.447436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c718b3-5bdb-4af4-9681-716c340b0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:18.447436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c718b3-5bdb-4af4-9681-716c340b0057 admin admin] PATH_INFO: `/v3/projects/77ebde9fdeef450f884905d8dff31900/users/d249a356ebd54bf3b5240f5b372bd61f/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:18.448589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c718b3-5bdb-4af4-9681-716c340b0057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ebde9fdeef450f884905d8dff31900, user_id=d249a356ebd54bf3b5240f5b372bd61f, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:18.449346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:18.453170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c718b3-5bdb-4af4-9681-716c340b0057 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:18.509925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 859/1684] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:18.510404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 826/1685] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:18 2022] PUT /identity/v3/projects/77ebde9fdeef450f884905d8dff31900/users/d249a356ebd54bf3b5240f5b372bd61f/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:18.520217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:18.549394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:18.549770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:18.550151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:18.572531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89fab7-3fe6-40b8-aa26-3aac64cbba3d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:18.706716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] MFA Rules not processed for user `d249a356ebd54bf3b5240f5b372bd61f`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:18.706716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:18.833491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb15d8-c59a-448c-97ac-f0fc1e544e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:18.834501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb15d8-c59a-448c-97ac-f0fc1e544e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:18.834947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb15d8-c59a-448c-97ac-f0fc1e544e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:18.835381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb15d8-c59a-448c-97ac-f0fc1e544e8a admin admin] PATH_INFO: `/v3/projects/bc2e7b9b933343b29daac84008d5a390/users/09786dcdecd8492180855feb464ca369/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:18.874746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb15d8-c59a-448c-97ac-f0fc1e544e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc2e7b9b933343b29daac84008d5a390, user_id=09786dcdecd8492180855feb464ca369, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:18.877981 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb15d8-c59a-448c-97ac-f0fc1e544e8a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:18.956793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 827/1686] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:18.961442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:18.970721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:18.973787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:19.001842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 860/1687] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:18 2022] PUT /identity/v3/projects/bc2e7b9b933343b29daac84008d5a390/users/09786dcdecd8492180855feb464ca369/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:19.012512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eacbe4-4326-4986-9e48-c1988d2962ef None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:19.012975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eacbe4-4326-4986-9e48-c1988d2962ef None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:19.013452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eacbe4-4326-4986-9e48-c1988d2962ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:19.080704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-24eacbe4-4326-4986-9e48-c1988d2962ef None None] MFA Rules not processed for user `09786dcdecd8492180855feb464ca369`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:19.082353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-24eacbe4-4326-4986-9e48-c1988d2962ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:19.104046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb280d56-770e-494c-8cbb-58ea98f3c517 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:19.210679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:19.211236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:19.211612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:19.211958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:19.227517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 861/1688] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:19.233670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eacbe4-4326-4986-9e48-c1988d2962ef None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:19.349355 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:19.352086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:19.361411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:19.391700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 828/1689] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:19.400782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eacbe4-4326-4986-9e48-c1988d2962ef None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:19.409905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:19.410381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:19.410763 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:19.411856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:19.413551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:19.416509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:19.509782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:19.515716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 862/1690] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:19 2022] POST /identity/v3/projects => generated 379 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:00:19.520793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:19.553576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d877-d135-41a4-aafa-bb19328f08ac service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:19.722897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:19.724539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:19.724539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:19.724539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:19.739869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:19.742036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:19.753255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:19.843743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb94950-19e9-4e37-8ffe-987136d926b8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:19.860073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741db815-cc89-4bb0-b43e-780e8a3062fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:19.860867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741db815-cc89-4bb0-b43e-780e8a3062fb service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:19.861194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741db815-cc89-4bb0-b43e-780e8a3062fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:19.861642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741db815-cc89-4bb0-b43e-780e8a3062fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:19.870974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 829/1691] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:19 2022] POST /identity/v3/users => generated 296 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:19.879964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:19.895434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5e1d8-0b37-4129-87d0-b5f35e65808e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:19.916356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:19.917955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:19.918391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:19.918760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:19.920670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:19.924576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:19.948160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 830/1692] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:19.956624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:19.979166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7d36c-4bf1-4ae1-bcfd-39dd36d01932 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:19.987941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:19.988928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:19.989749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:19.990233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] PATH_INFO: `/v3/projects/040a98929ff44bffb7454f29639ca1de/users/05b8599465864f9ab39a20d1c76b1494/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:20.035129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040a98929ff44bffb7454f29639ca1de, user_id=05b8599465864f9ab39a20d1c76b1494, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:20.042405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:20.087519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741db815-cc89-4bb0-b43e-780e8a3062fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:20.089308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741db815-cc89-4bb0-b43e-780e8a3062fb service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:20.137517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 831/1693] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:19 2022] PUT /identity/v3/projects/040a98929ff44bffb7454f29639ca1de/users/05b8599465864f9ab39a20d1c76b1494/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:20.141981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:20.146335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 863/1694] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:20.181901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:20.182364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:20.192090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:20.270810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:20.317703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] MFA Rules not processed for user `0d2c7d95047445ee9d44640b5372e0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:20.320659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:20.344381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06fe5-c65b-4db3-8aee-0cdb9bf060b6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:20.358975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:20.359660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:20.359950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:20.360224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:20.362185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:20.366038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:20.413304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 832/1695] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:20.414240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:20.466197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc66409-e471-40b0-afae-5c9de32018a5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:20.489855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4394bf4-ce64-4ddd-9fd1-4fc4c3ac2fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:20.492056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4394bf4-ce64-4ddd-9fd1-4fc4c3ac2fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:20.492433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4394bf4-ce64-4ddd-9fd1-4fc4c3ac2fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:20.502413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4394bf4-ce64-4ddd-9fd1-4fc4c3ac2fbd admin admin] PATH_INFO: `/v3/projects/040a98929ff44bffb7454f29639ca1de/users/05b8599465864f9ab39a20d1c76b1494/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:20.524003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4394bf4-ce64-4ddd-9fd1-4fc4c3ac2fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=040a98929ff44bffb7454f29639ca1de, user_id=05b8599465864f9ab39a20d1c76b1494, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:20.524219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 864/1696] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:20.524710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:20.527603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4394bf4-ce64-4ddd-9fd1-4fc4c3ac2fbd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:20.533807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:20.537144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:20.585925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 833/1697] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:20 2022] PUT /identity/v3/projects/040a98929ff44bffb7454f29639ca1de/users/05b8599465864f9ab39a20d1c76b1494/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:20.585925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdd706-1150-48e5-8a2e-bb3911d4270f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:20.585925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdd706-1150-48e5-8a2e-bb3911d4270f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:20.591028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdd706-1150-48e5-8a2e-bb3911d4270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:20.700303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b5bdd706-1150-48e5-8a2e-bb3911d4270f None None] MFA Rules not processed for user `05b8599465864f9ab39a20d1c76b1494`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:20.706061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b5bdd706-1150-48e5-8a2e-bb3911d4270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:20.843641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276293a-5372-4c8d-a653-21cfc3f437dd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:20.959835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 834/1698] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:21.031330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdd706-1150-48e5-8a2e-bb3911d4270f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:21.122130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef1f55-577e-41a9-8572-6435c2d402e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:21.124039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef1f55-577e-41a9-8572-6435c2d402e2 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:21.124566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef1f55-577e-41a9-8572-6435c2d402e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:21.125649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef1f55-577e-41a9-8572-6435c2d402e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:21.203291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdd706-1150-48e5-8a2e-bb3911d4270f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:21.215956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:21.216756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:21.217054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:21.217329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:21.240526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:21.242172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:21.272901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 835/1699] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:21.306037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef1f55-577e-41a9-8572-6435c2d402e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:21.309853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef1f55-577e-41a9-8572-6435c2d402e2 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:21.351455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 865/1700] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 818 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:21.657892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:21.892670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:22.209088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:22.398486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab1af-3f46-4710-a7ac-4267db4d7ac2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:22.399631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab1af-3f46-4710-a7ac-4267db4d7ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:22.400113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab1af-3f46-4710-a7ac-4267db4d7ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:22.454107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8d540-a2f8-4640-ad32-2b963fd3d815 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:22.467560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:22.471817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:22.471817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:22.471817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] PATH_INFO: `/v3/users/80d2204c01aa4a40b26517d7b80d19b7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:22.471817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d2204c01aa4a40b26517d7b80d19b7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:22.473676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:22.474390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fe6ab1af-3f46-4710-a7ac-4267db4d7ac2 None None] MFA Rules not processed for user `f030155981d741f1a42b9b84ca5d6af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:22.476326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fe6ab1af-3f46-4710-a7ac-4267db4d7ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:22.692249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 866/1701] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:22.698554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c337-08b7-4774-b7dd-0b2a6bab2bda None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:22.698847 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c337-08b7-4774-b7dd-0b2a6bab2bda None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:22.699129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c337-08b7-4774-b7dd-0b2a6bab2bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:22.721228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d2204c01aa4a40b26517d7b80d19b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:22.766457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d2204c01aa4a40b26517d7b80d19b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:22.795346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-65c5c337-08b7-4774-b7dd-0b2a6bab2bda None None] MFA Rules not processed for user `972bc4e9bc9e49b4a3bc2a693b05e4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:22.798465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-65c5c337-08b7-4774-b7dd-0b2a6bab2bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:22.891652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d2204c01aa4a40b26517d7b80d19b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:22.892667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d2204c01aa4a40b26517d7b80d19b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:22.920150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d2204c01aa4a40b26517d7b80d19b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:22.940969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 836/1702] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:22 2022] DELETE /identity/v3/users/80d2204c01aa4a40b26517d7b80d19b7 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:22.944671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:22.947442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:22.970719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 867/1703] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:22.972645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393b278-3e18-4462-9bdc-b08a72967721 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:22.973106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393b278-3e18-4462-9bdc-b08a72967721 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:22.973608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393b278-3e18-4462-9bdc-b08a72967721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:23.000834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:23.023931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c393b278-3e18-4462-9bdc-b08a72967721 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:23.026105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c393b278-3e18-4462-9bdc-b08a72967721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:23.140704 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcadcc-2667-41d3-bb59-b23508572ccc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:23.148117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:23.149836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:23.150329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:23.150850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:23.159759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:23.199929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 868/1704] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:23.207007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:23.207007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:23.207007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:23.238420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] MFA Rules not processed for user `972bc4e9bc9e49b4a3bc2a693b05e4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:23.240197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:23.325454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:23.327500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:23.344207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 837/1705] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:23.345327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:23.346981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:23.447170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 869/1706] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:23.567170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c28ae-2f55-4298-a910-cf1a4d5d2d25 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:23.583042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:23.584819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:23.584819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:23.588458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:23.589516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:23.594569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:23.624832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e8849-b1f5-42c5-8559-03a6e9440ba1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:23.646796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:23.647992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:23.648338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:23.648830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:23.653784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:23.659146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:23.689243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:23.693895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:23.709320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 838/1707] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:23.746057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 870/1708] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:23.756940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:23.758659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:24.018574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a485d8e-b605-4820-8425-4779214d1cf8 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:24.037354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b8599465864f9ab39a20d1c76b1494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '040a98929ff44bffb7454f29639ca1de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:24.038198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:24.038675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:24.039092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:24.040077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:24.040657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:24.041415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:24.043329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:24.081895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 839/1709] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:24.086164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:24.108582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0b918-04c6-4ad5-9a0e-1b1ab226ffe1 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:24.124009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea6683-3d7b-494d-b363-b431f7a0b39e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b8599465864f9ab39a20d1c76b1494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '040a98929ff44bffb7454f29639ca1de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:24.124921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6683-3d7b-494d-b363-b431f7a0b39e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:24.125553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6683-3d7b-494d-b363-b431f7a0b39e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:24.125990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6683-3d7b-494d-b363-b431f7a0b39e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:24.127870 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea6683-3d7b-494d-b363-b431f7a0b39e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:24.132534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea6683-3d7b-494d-b363-b431f7a0b39e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:24.342724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 871/1710] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:24 2022] POST /identity/v3/projects => generated 353 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:24.357882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:24.370496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9a19-8948-4c2d-aaf2-6e1ad74c68b2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:24.371274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9a19-8948-4c2d-aaf2-6e1ad74c68b2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:24.371750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9a19-8948-4c2d-aaf2-6e1ad74c68b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:24.380470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a19ffb-38c2-415d-adbc-f46ae250142e tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:24.471897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e21c9a19-8948-4c2d-aaf2-6e1ad74c68b2 None None] MFA Rules not processed for user `570de5da908c4b31a8a22706642aafb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:24.471897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e21c9a19-8948-4c2d-aaf2-6e1ad74c68b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:24.582855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4d34d-4e6b-452d-9ce4-c62068c1fedf tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b8599465864f9ab39a20d1c76b1494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '040a98929ff44bffb7454f29639ca1de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:24.583506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4d34d-4e6b-452d-9ce4-c62068c1fedf tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:24.584054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4d34d-4e6b-452d-9ce4-c62068c1fedf tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:24.584156 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4d34d-4e6b-452d-9ce4-c62068c1fedf tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:24.584794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4d34d-4e6b-452d-9ce4-c62068c1fedf tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:24.585109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4d34d-4e6b-452d-9ce4-c62068c1fedf tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:24.586768 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4d34d-4e6b-452d-9ce4-c62068c1fedf tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:24.645874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 840/1711] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:24.714658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea539af-277e-4a74-b772-9aa1132b9af0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:24.714958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea539af-277e-4a74-b772-9aa1132b9af0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:24.715245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea539af-277e-4a74-b772-9aa1132b9af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:24.715403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 872/1712] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:24.789382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-aea539af-277e-4a74-b772-9aa1132b9af0 None None] MFA Rules not processed for user `570de5da908c4b31a8a22706642aafb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:24.790645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-aea539af-277e-4a74-b772-9aa1132b9af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:24.815858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 841/1713] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:24.828140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c9a19-8948-4c2d-aaf2-6e1ad74c68b2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:24.873349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea539af-277e-4a74-b772-9aa1132b9af0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:24.910739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:24.923606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea539af-277e-4a74-b772-9aa1132b9af0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:24.937539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05b8599465864f9ab39a20d1c76b1494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '040a98929ff44bffb7454f29639ca1de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:24.942524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:24.942827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:24.943121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] PATH_INFO: `/v3/projects/10325598e32848c7aa20f4792db4373b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:24.954135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorizing `identity:delete_project(project_id=10325598e32848c7aa20f4792db4373b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:24.961940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:25.117755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c9a19-8948-4c2d-aaf2-6e1ad74c68b2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:25.135843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07774888-887c-4e46-af30-563d6bfbf8d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:25.136674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07774888-887c-4e46-af30-563d6bfbf8d2 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:25.137104 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07774888-887c-4e46-af30-563d6bfbf8d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:25.137545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07774888-887c-4e46-af30-563d6bfbf8d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:25.160332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07774888-887c-4e46-af30-563d6bfbf8d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:25.162172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07774888-887c-4e46-af30-563d6bfbf8d2 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:25.200506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:25.207524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 873/1714] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:25.212394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8d048-a82f-484a-b083-09fd503dfb29 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:25.214293 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8d048-a82f-484a-b083-09fd503dfb29 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:25.217191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8d048-a82f-484a-b083-09fd503dfb29 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:25.224222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 874/1715] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 08:00:25 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:25.232295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8d048-a82f-484a-b083-09fd503dfb29 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:25.242650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a6f8d048-a82f-484a-b083-09fd503dfb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:25.300301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10325598e32848c7aa20f4792db4373b', 'request_id': 'req-9b17d33e-ed9d-4338-aaa5-511b512ad16b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.338557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10325598e32848c7aa20f4792db4373b', 'request_id': 'req-9b17d33e-ed9d-4338-aaa5-511b512ad16b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.364742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10325598e32848c7aa20f4792db4373b', 'request_id': 'req-9b17d33e-ed9d-4338-aaa5-511b512ad16b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.478501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8d048-a82f-484a-b083-09fd503dfb29 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:25.488273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:25.489029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:25.489408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:25.489770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] PATH_INFO: `/v3/projects/8b04028ada0b456c89dc71071a51f57e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:25.491209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b04028ada0b456c89dc71071a51f57e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:25.493671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 08:00:25.493671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: warnings.warn(msg) Aug 01 08:00:25.493671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: Aug 01 08:00:25.502837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] The token cache is being invalidate because project 10325598e32848c7aa20f4792db4373b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:25.505116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.505116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9b17d33e-ed9d-4338-aaa5-511b512ad16b tempest-VolumeQuotaClassesTest-292768435 tempest-VolumeQuotaClassesTest-292768435-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.509295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 842/1716] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:24 2022] DELETE /identity/v3/projects/10325598e32848c7aa20f4792db4373b => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:25.530211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:25.532626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:25.533055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:25.533565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:25.672497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] MFA Rules not processed for user `ad3cf8a3e0a346fa867445d6b6300447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:25.675185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:25.770731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b04028ada0b456c89dc71071a51f57e', 'request_id': 'req-0a11cdc7-8614-45ff-b868-bd1a90e21254'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.795679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 843/1717] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:00:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:25.844877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:25.846506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b04028ada0b456c89dc71071a51f57e', 'request_id': 'req-0a11cdc7-8614-45ff-b868-bd1a90e21254'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.849106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:25.859228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b04028ada0b456c89dc71071a51f57e', 'request_id': 'req-0a11cdc7-8614-45ff-b868-bd1a90e21254'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.873083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab8245-840a-4882-a4ff-2ebb4c584bb8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:25.926025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] The token cache is being invalidate because project 8b04028ada0b456c89dc71071a51f57e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:25.926347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.929965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:25.937140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 875/1718] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:25 2022] DELETE /identity/v3/projects/8b04028ada0b456c89dc71071a51f57e => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:25.943042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:25.948748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:25.973509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3d656-1fb5-487a-9ef0-b7f1d415f103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:25.974487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3d656-1fb5-487a-9ef0-b7f1d415f103 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:25.977139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3d656-1fb5-487a-9ef0-b7f1d415f103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:25.977541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3d656-1fb5-487a-9ef0-b7f1d415f103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:25.984900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-70a3d656-1fb5-487a-9ef0-b7f1d415f103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:26.011891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3d656-1fb5-487a-9ef0-b7f1d415f103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:26.019111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3d656-1fb5-487a-9ef0-b7f1d415f103 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:26.042497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 844/1719] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:26.062726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df93826-34e0-4cae-8ce9-7fc9ddfb1186 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:26.063078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df93826-34e0-4cae-8ce9-7fc9ddfb1186 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:26.063472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df93826-34e0-4cae-8ce9-7fc9ddfb1186 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:26.065789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 845/1720] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 08:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:26.076934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11cdc7-8614-45ff-b868-bd1a90e21254 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:26.077136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:26.077460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:26.077981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:26.102442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:26.103406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:26.105094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:26.105835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] PATH_INFO: `/v3/users/d249a356ebd54bf3b5240f5b372bd61f` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:26.108061 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d249a356ebd54bf3b5240f5b372bd61f)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:26.111790 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:26.145752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] MFA Rules not processed for user `ad3cf8a3e0a346fa867445d6b6300447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:26.148710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:26.164427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 846/1721] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:00:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:26.185540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:26.185540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:26.197865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa0a17-b4b2-4098-8f1d-3b1686d85a09 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:26.202926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:26.203958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:26.204431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:26.204633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:26.216201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:26.217812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:26.228621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 847/1722] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:26.400395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:26.402256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:26.410640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd249a356ebd54bf3b5240f5b372bd61f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.439471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475a2e3-e8ea-4ede-8275-eb6c808a2e0f service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:26.455595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:26.456465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:26.456756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:26.457034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] PATH_INFO: `/v3/users/09786dcdecd8492180855feb464ca369` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:26.462717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09786dcdecd8492180855feb464ca369)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:26.472061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:26.505259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd249a356ebd54bf3b5240f5b372bd61f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.515843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd249a356ebd54bf3b5240f5b372bd61f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.517470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd249a356ebd54bf3b5240f5b372bd61f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.522478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-15f16f47-e288-46ce-979b-baf1a20b8b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd249a356ebd54bf3b5240f5b372bd61f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.573875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 876/1723] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:25 2022] DELETE /identity/v3/users/d249a356ebd54bf3b5240f5b372bd61f => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:26.578154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:26.788886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09786dcdecd8492180855feb464ca369'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.807521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09786dcdecd8492180855feb464ca369'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.892423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09786dcdecd8492180855feb464ca369'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.895578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09786dcdecd8492180855feb464ca369'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.958801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09786dcdecd8492180855feb464ca369'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:26.979252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 848/1724] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:26 2022] DELETE /identity/v3/users/09786dcdecd8492180855feb464ca369 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:26.984722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd568-e4b3-4fb8-8161-851c22ffedae None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:26.985749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd568-e4b3-4fb8-8161-851c22ffedae None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:26.986175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd568-e4b3-4fb8-8161-851c22ffedae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.040561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d7dfd568-e4b3-4fb8-8161-851c22ffedae None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:27.041815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d7dfd568-e4b3-4fb8-8161-851c22ffedae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:27.109744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:27.205092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 877/1725] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:27.272503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:27.274542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:27.366124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1c0ba-1097-4ae7-8b8f-8f951f35a8dc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:27.366524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1c0ba-1097-4ae7-8b8f-8f951f35a8dc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:27.366872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1c0ba-1097-4ae7-8b8f-8f951f35a8dc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.368589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 878/1726] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 1 08:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:27.371346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1c0ba-1097-4ae7-8b8f-8f951f35a8dc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:27.373739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-84f1c0ba-1097-4ae7-8b8f-8f951f35a8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:27.416301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86884eff-968a-4269-8ec3-6bea114fb331 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:27.433292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be4745-7007-42d1-a8c4-0c4ed1710ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:27.434025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be4745-7007-42d1-a8c4-0c4ed1710ded service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:27.434318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be4745-7007-42d1-a8c4-0c4ed1710ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:27.435230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be4745-7007-42d1-a8c4-0c4ed1710ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.442018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:27.482041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be4745-7007-42d1-a8c4-0c4ed1710ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:27.483724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be4745-7007-42d1-a8c4-0c4ed1710ded service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:27.511709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 849/1727] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:27.521149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2869-dc8a-4eab-9363-5643a886f943 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:27.521685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2869-dc8a-4eab-9363-5643a886f943 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:27.522215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2869-dc8a-4eab-9363-5643a886f943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.568323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b4ec2869-dc8a-4eab-9363-5643a886f943 None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:27.571877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b4ec2869-dc8a-4eab-9363-5643a886f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:27.581144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1c0ba-1097-4ae7-8b8f-8f951f35a8dc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:27.589684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 850/1728] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 1 08:00:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:27.596770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:27.599367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:27.599779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:27.599893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:27.600068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.601144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:27.602651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.609055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:27.624365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] MFA Rules not processed for user `972bc4e9bc9e49b4a3bc2a693b05e4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:27.625712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:27.642558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:27.647567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:27.690266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 879/1729] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:27.691186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:27.710614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06322208-4518-4347-835a-e05138fc2fd9 service placement] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:27.714496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 851/1730] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:27.723941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be8cc9-6bca-4c2a-8aa3-72b64d4b6350 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:27.724584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8cc9-6bca-4c2a-8aa3-72b64d4b6350 service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:27.725075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8cc9-6bca-4c2a-8aa3-72b64d4b6350 service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:27.725186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8cc9-6bca-4c2a-8aa3-72b64d4b6350 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.755837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be8cc9-6bca-4c2a-8aa3-72b64d4b6350 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:27.755837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be8cc9-6bca-4c2a-8aa3-72b64d4b6350 service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:27.813718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:27.813718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:27.831510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 880/1731] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:27.867895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a37ea6-995e-416b-84b4-2bf70eda84e8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:27.885321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:27.892111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:27.893953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:27.894605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] PATH_INFO: `/v3/projects/77ebde9fdeef450f884905d8dff31900` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:27.900132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ebde9fdeef450f884905d8dff31900)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:27.905908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:28.260860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ebde9fdeef450f884905d8dff31900', 'request_id': 'req-a4e40af3-74ea-4653-883c-47dd6fa65bc7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:28.272891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ebde9fdeef450f884905d8dff31900', 'request_id': 'req-a4e40af3-74ea-4653-883c-47dd6fa65bc7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:28.301030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ebde9fdeef450f884905d8dff31900', 'request_id': 'req-a4e40af3-74ea-4653-883c-47dd6fa65bc7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:28.421366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] The token cache is being invalidate because project 77ebde9fdeef450f884905d8dff31900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:28.421366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:28.422992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4e40af3-74ea-4653-883c-47dd6fa65bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:28.431058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 852/1732] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:27 2022] DELETE /identity/v3/projects/77ebde9fdeef450f884905d8dff31900 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:28.433067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae024d8-92e0-47cb-8793-8572f27ab54b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:28.433538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae024d8-92e0-47cb-8793-8572f27ab54b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:28.433812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae024d8-92e0-47cb-8793-8572f27ab54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:28.498009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6ae024d8-92e0-47cb-8793-8572f27ab54b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:28.503381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6ae024d8-92e0-47cb-8793-8572f27ab54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:28.698668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 881/1733] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:28.707345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63200daa-406d-4eaf-8e9d-23b9972e8011 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:28.707763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63200daa-406d-4eaf-8e9d-23b9972e8011 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:28.708223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63200daa-406d-4eaf-8e9d-23b9972e8011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:28.738673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-63200daa-406d-4eaf-8e9d-23b9972e8011 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:28.740499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-63200daa-406d-4eaf-8e9d-23b9972e8011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:28.765771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 853/1734] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:28.769962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae024d8-92e0-47cb-8793-8572f27ab54b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:28.785795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae024d8-92e0-47cb-8793-8572f27ab54b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:28.795471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:28.796342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:28.796776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:28.799453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:28.799453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:28.799453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:28.808417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:28.833542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 882/1735] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:28.837632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63200daa-406d-4eaf-8e9d-23b9972e8011 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:28.880314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63200daa-406d-4eaf-8e9d-23b9972e8011 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:28.905890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:28.906864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:28.916953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:28.920223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:28.920223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:28.921356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:29.059622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 854/1736] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:28 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:29.063208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:29.079806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fc516-8fb0-4462-9c35-47d76ea361b2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:29.193989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52e79-8906-41f1-aa76-d29ac08a916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:29.194820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52e79-8906-41f1-aa76-d29ac08a916e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:29.196072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52e79-8906-41f1-aa76-d29ac08a916e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:29.196495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52e79-8906-41f1-aa76-d29ac08a916e admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:29.197901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe52e79-8906-41f1-aa76-d29ac08a916e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:29.201311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe52e79-8906-41f1-aa76-d29ac08a916e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:29.330897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 883/1737] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:29 2022] POST /identity/v3/users => generated 288 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:29.334734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:29.353511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2f63-3bd8-48bc-a56b-a7759a5270f0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:29.361185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:29.362450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:29.363675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:29.364732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:29.367723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:29.375977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:29.380543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:29.381124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:29.381611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:29.413668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 855/1738] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:29.420907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:29.433217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] MFA Rules not processed for user `0d2c7d95047445ee9d44640b5372e0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:29.435635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e623ff5-927d-4f84-9809-10ba7837158c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:29.439174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:29.442722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:29.449614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:29.449614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:29.449614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:29.449614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:29.449614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:29.467025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 856/1739] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:29.469860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:29.485016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f9e4e-84e1-4333-a1cc-44b4ef063fa7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:29.492106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:29.493555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:29.494035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:29.494539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] PATH_INFO: `/v3/projects/5fc90c87605841edb097fe3b12874712/users/87cb3e7f265940e19220664fe7c40b36/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:29.518690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc90c87605841edb097fe3b12874712, user_id=87cb3e7f265940e19220664fe7c40b36, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:29.521727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:29.589158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 884/1740] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:29.590482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:29.605235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:29.630627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6fe71-c0a6-4f91-a6dd-5ca209493802 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:29.637868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:29.638796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:29.639233 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:29.639769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] PATH_INFO: `/v3/users/7280265a21ab4cfda4538a3064ab8853` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:29.667830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7280265a21ab4cfda4538a3064ab8853)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:29.669263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:29.703273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 857/1741] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:29 2022] PUT /identity/v3/projects/5fc90c87605841edb097fe3b12874712/users/87cb3e7f265940e19220664fe7c40b36/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:29.721998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:29.729988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:29.901694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7280265a21ab4cfda4538a3064ab8853'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:29.981032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c237a-f6db-4b65-8918-2dac9c5503b8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:29.986138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:30.010046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:30.010046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:30.010046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:30.010046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] PATH_INFO: `/v3/projects/bc2e7b9b933343b29daac84008d5a390` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:30.010384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc2e7b9b933343b29daac84008d5a390)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:30.010384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:30.030688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7280265a21ab4cfda4538a3064ab8853'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.053856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7280265a21ab4cfda4538a3064ab8853'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.054878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7280265a21ab4cfda4538a3064ab8853'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.055698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cbb11ade-ef29-43c2-a8f1-4da16dd29490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7280265a21ab4cfda4538a3064ab8853'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.133582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 885/1742] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:29 2022] DELETE /identity/v3/users/7280265a21ab4cfda4538a3064ab8853 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:30.135362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d1315-ead1-4537-949f-bea6dcb99137 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:30.136683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d1315-ead1-4537-949f-bea6dcb99137 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:30.140676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d1315-ead1-4537-949f-bea6dcb99137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:30.211039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e98d1315-ead1-4537-949f-bea6dcb99137 None None] MFA Rules not processed for user `87cb3e7f265940e19220664fe7c40b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:30.216723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e98d1315-ead1-4537-949f-bea6dcb99137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:30.233252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc2e7b9b933343b29daac84008d5a390', 'request_id': 'req-24a2b782-f918-4a3c-a8a0-b1f641230d92'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.257638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc2e7b9b933343b29daac84008d5a390', 'request_id': 'req-24a2b782-f918-4a3c-a8a0-b1f641230d92'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.301536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc2e7b9b933343b29daac84008d5a390', 'request_id': 'req-24a2b782-f918-4a3c-a8a0-b1f641230d92'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.360987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 886/1743] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:30.363431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:30.366964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:30.368113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:30.408610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] The token cache is being invalidate because project bc2e7b9b933343b29daac84008d5a390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:30.408610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.408610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.408905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:30.412917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 858/1744] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:29 2022] DELETE /identity/v3/projects/bc2e7b9b933343b29daac84008d5a390 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:30.412917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:30.413148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:30.413232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:30.482915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:30.595783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 887/1745] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:30.597458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:30.602674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:30.625838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2b782-f918-4a3c-a8a0-b1f641230d92 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:30.631349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db0bc7-59f5-4214-84e9-451803266502 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:30.638315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:30.638972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:30.639258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:30.639553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:30.641623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:30.644549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:30.645653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:30.647448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:30.647728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:30.648155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] PATH_INFO: `/v3/users/05b8599465864f9ab39a20d1c76b1494` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:30.650782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b8599465864f9ab39a20d1c76b1494)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:30.652943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:30.826873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 859/1746] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:30 2022] POST /identity/v3/projects => generated 375 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:30.828507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:30.830472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:30.917278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b8599465864f9ab39a20d1c76b1494'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:30.959317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0caa8d-7554-4ce3-8d1b-954d6ef51d61 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:30.964493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91609966-6d58-4f31-8bb5-f85c4ad20444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:30.967298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91609966-6d58-4f31-8bb5-f85c4ad20444 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:30.967629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91609966-6d58-4f31-8bb5-f85c4ad20444 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:30.967918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91609966-6d58-4f31-8bb5-f85c4ad20444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.016614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b8599465864f9ab39a20d1c76b1494'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:31.035082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b8599465864f9ab39a20d1c76b1494'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:31.064042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b8599465864f9ab39a20d1c76b1494'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:31.066703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b8599465864f9ab39a20d1c76b1494'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:31.108447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 888/1747] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:30 2022] DELETE /identity/v3/users/05b8599465864f9ab39a20d1c76b1494 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:31.110634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:31.112625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:31.122469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:31.179855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91609966-6d58-4f31-8bb5-f85c4ad20444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:31.181329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91609966-6d58-4f31-8bb5-f85c4ad20444 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:31.197067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 860/1748] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:31.202056 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b7885-3e2b-4252-8698-f3c056f235fb None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:31.203292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b7885-3e2b-4252-8698-f3c056f235fb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:31.203760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b7885-3e2b-4252-8698-f3c056f235fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.265639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e72b7885-3e2b-4252-8698-f3c056f235fb None None] MFA Rules not processed for user `f030155981d741f1a42b9b84ca5d6af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:31.273676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e72b7885-3e2b-4252-8698-f3c056f235fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:31.284109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1ae99-be23-4b79-beb7-44b17fdd1a66 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:31.290391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:31.291335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:31.291845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:31.292644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.293564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:31.299186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:31.442107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 889/1749] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:31 2022] POST /identity/v3/users => generated 288 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:31.443328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:31.465006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 861/1750] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:31.495941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25269a-02ab-4424-8215-fb72adcab063 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:31.503228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:31.503951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:31.507379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:31.507379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.507379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:31.507379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:31.523662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 890/1751] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:31.526364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b7885-3e2b-4252-8698-f3c056f235fb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:31.540025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b7885-3e2b-4252-8698-f3c056f235fb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:31.546132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9d757-1364-449c-9f88-e47d9fc223d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:31.546774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9d757-1364-449c-9f88-e47d9fc223d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:31.547062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9d757-1364-449c-9f88-e47d9fc223d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:31.547343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9d757-1364-449c-9f88-e47d9fc223d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.548493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9d757-1364-449c-9f88-e47d9fc223d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:31.550208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9d757-1364-449c-9f88-e47d9fc223d1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:31.562658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 862/1752] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:31.566773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:31.594176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7333-ac7e-4266-a371-0ce12015c0a7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:31.601274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:31.602183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:31.602473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:31.602759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] PATH_INFO: `/v3/projects/5c72a36a57094831aef34a02ba4c89e2/users/a149c3b7c6f549dc92d42ba5019e59a1/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.627609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c72a36a57094831aef34a02ba4c89e2, user_id=a149c3b7c6f549dc92d42ba5019e59a1, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:31.627746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:31.720679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 891/1753] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:31 2022] PUT /identity/v3/projects/5c72a36a57094831aef34a02ba4c89e2/users/a149c3b7c6f549dc92d42ba5019e59a1/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:31.734976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:31.735800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:31.736223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.783199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] MFA Rules not processed for user `a149c3b7c6f549dc92d42ba5019e59a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:31.783199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:31.885366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 863/1754] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:31.890566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:31.990319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b270c-07aa-4e32-b110-8ecb85f9550f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:31.997215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:31.997890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:31.998295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:31.998691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:31.999416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:32.001430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:32.057837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:32.059929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:32.079764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98a7e-9628-478d-8101-483e5458f982 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:32.085520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:32.086442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:32.086802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:32.087277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] PATH_INFO: `/v3/projects/040a98929ff44bffb7454f29639ca1de` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:32.088771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=040a98929ff44bffb7454f29639ca1de)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:32.091452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:32.106551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 892/1755] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:31 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:00:32.123701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:32.208659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:32.296160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040a98929ff44bffb7454f29639ca1de', 'request_id': 'req-44428209-bff1-4969-ac9a-24dfbab348a6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.305215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748145f-ba2d-4d04-8ef0-f8cf97ab8653 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:32.306619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040a98929ff44bffb7454f29639ca1de', 'request_id': 'req-44428209-bff1-4969-ac9a-24dfbab348a6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.368525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040a98929ff44bffb7454f29639ca1de', 'request_id': 'req-44428209-bff1-4969-ac9a-24dfbab348a6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.428782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a2e63-0478-461a-8874-db5885ad2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:32.430305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2e63-0478-461a-8874-db5885ad2251 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:32.430598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2e63-0478-461a-8874-db5885ad2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:32.430878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2e63-0478-461a-8874-db5885ad2251 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:32.431577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a2e63-0478-461a-8874-db5885ad2251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:32.435360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a2e63-0478-461a-8874-db5885ad2251 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:32.534093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] The token cache is being invalidate because project 040a98929ff44bffb7454f29639ca1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:32.534404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.535065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.544658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 864/1756] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:32 2022] DELETE /identity/v3/projects/040a98929ff44bffb7454f29639ca1de => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:32.545760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:32.548828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:32.567775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428209-bff1-4969-ac9a-24dfbab348a6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:32.574634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:32.575681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:32.586411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 893/1757] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:32 2022] POST /identity/v3/users => generated 295 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:00:32.590154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:32.590154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] PATH_INFO: `/v3/projects/9d14e0b976344ea7a152250846d587dd` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:32.590761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d14e0b976344ea7a152250846d587dd)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:32.592616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:32.593293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:32.593460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:32.594655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:32.635956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:32.637231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:32.657309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 894/1758] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:32.658314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:32.660113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:32.674820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887c271-e7d2-4acc-bbf8-2c35e3922dce None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:32.680643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7433d8a-4cf4-4a7c-85ee-36471ceeaf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:32.682721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7433d8a-4cf4-4a7c-85ee-36471ceeaf4e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:32.683555 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7433d8a-4cf4-4a7c-85ee-36471ceeaf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:32.683555 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7433d8a-4cf4-4a7c-85ee-36471ceeaf4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:32.684858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7433d8a-4cf4-4a7c-85ee-36471ceeaf4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:32.686663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7433d8a-4cf4-4a7c-85ee-36471ceeaf4e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:32.703747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 895/1759] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:32.705109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1166d-0bfd-4e63-b746-7d0b17434256 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:32.706719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1166d-0bfd-4e63-b746-7d0b17434256 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:32.706719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1166d-0bfd-4e63-b746-7d0b17434256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:32.755780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-37a1166d-0bfd-4e63-b746-7d0b17434256 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:32.757160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-37a1166d-0bfd-4e63-b746-7d0b17434256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:32.786354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 896/1760] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:32.787903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1166d-0bfd-4e63-b746-7d0b17434256 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:32.812819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1166d-0bfd-4e63-b746-7d0b17434256 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:32.820366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d14e0b976344ea7a152250846d587dd', 'request_id': 'req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.842243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d14e0b976344ea7a152250846d587dd', 'request_id': 'req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.868979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d14e0b976344ea7a152250846d587dd', 'request_id': 'req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.970029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:32.971205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:32.971799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:32.972281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] PATH_INFO: `/v3/projects/099e35d20cd94455988579feb25288c9/users/3835260bf10c4ab7b4f1b1ebfee9b1f9/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:32.996784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] The token cache is being invalidate because project 9d14e0b976344ea7a152250846d587dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:32.996945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:32.998186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:33.001575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 865/1761] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:32 2022] DELETE /identity/v3/projects/9d14e0b976344ea7a152250846d587dd => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:33.002745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:33.004372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:33.042993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0a04-3410-41d7-bfd8-ec400dd4c9c7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:33.048929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] RBAC: Authorizing `identity:create_grant(project_id=099e35d20cd94455988579feb25288c9, user_id=3835260bf10c4ab7b4f1b1ebfee9b1f9, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:33.062840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:33.064099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb27ce-c3a0-4159-8377-df2337fdd77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:33.066262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb27ce-c3a0-4159-8377-df2337fdd77e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.066708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb27ce-c3a0-4159-8377-df2337fdd77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.067000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb27ce-c3a0-4159-8377-df2337fdd77e admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.068137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb27ce-c3a0-4159-8377-df2337fdd77e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:33.068442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb27ce-c3a0-4159-8377-df2337fdd77e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:33.070149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb27ce-c3a0-4159-8377-df2337fdd77e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:33.106510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 866/1762] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:33.109592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.109829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.110119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.179483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 897/1763] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:32 2022] PUT /identity/v3/projects/099e35d20cd94455988579feb25288c9/users/3835260bf10c4ab7b4f1b1ebfee9b1f9/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:33.180677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:33.202340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:33.203788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:33.368628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5073df-dc68-4be7-89e4-9e72e5b5f72e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:33.381753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c0f19-d679-4cac-b4fe-64011fee428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:33.384326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0f19-d679-4cac-b4fe-64011fee428a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.384882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0f19-d679-4cac-b4fe-64011fee428a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.385500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0f19-d679-4cac-b4fe-64011fee428a admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.386394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c0f19-d679-4cac-b4fe-64011fee428a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:33.389895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c0f19-d679-4cac-b4fe-64011fee428a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:33.489167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 867/1764] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:33.490835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:33.492610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:33.570916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df9a43-dc36-4d13-ae1b-fbc0c7c080d5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:33.578479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 898/1765] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:33 2022] POST /identity/v3/projects => generated 371 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:33.578479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf68a5a-3d68-4280-8fc3-2ad103d263e3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.578479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf68a5a-3d68-4280-8fc3-2ad103d263e3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.579209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf68a5a-3d68-4280-8fc3-2ad103d263e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.597561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:33.598476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.598948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.599796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.606185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:33.606185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:33.609695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-adf68a5a-3d68-4280-8fc3-2ad103d263e3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:33.610979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-adf68a5a-3d68-4280-8fc3-2ad103d263e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:33.631662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 868/1766] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:33.635101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:33.647905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d67e11-4d55-4483-a2cc-f2fa1827ded2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:33.724259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 899/1767] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:33.737744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:33.737917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac070b-25ba-4910-b5cb-118c7e676888 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.738302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac070b-25ba-4910-b5cb-118c7e676888 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.738589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac070b-25ba-4910-b5cb-118c7e676888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.739308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.739822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.740312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.741167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:33.744396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:33.832321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:33.874303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fdac070b-25ba-4910-b5cb-118c7e676888 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:33.875825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fdac070b-25ba-4910-b5cb-118c7e676888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:33.883115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 869/1768] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:33 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:33.886006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:33.921373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0eb41-e45e-41bb-9010-e624ecb2a89e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:33.939247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:33.940000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:33.940292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:33.940571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] PATH_INFO: `/v3/projects/099e35d20cd94455988579feb25288c9/users/3835260bf10c4ab7b4f1b1ebfee9b1f9/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:33.948865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=099e35d20cd94455988579feb25288c9, user_id=3835260bf10c4ab7b4f1b1ebfee9b1f9, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:33.969704 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:34.019982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 870/1769] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:33 2022] PUT /identity/v3/projects/099e35d20cd94455988579feb25288c9/users/3835260bf10c4ab7b4f1b1ebfee9b1f9/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:34.029102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:34.090058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 900/1770] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:00:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:34.091324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac070b-25ba-4910-b5cb-118c7e676888 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:34.110406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac070b-25ba-4910-b5cb-118c7e676888 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:34.212565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a1d6-29c0-44f1-aa0f-ab5a5fd57fb4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:34.229648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d8588-daf4-4817-9b6e-f017d971fc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:34.230417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8588-daf4-4817-9b6e-f017d971fc71 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:34.230636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8588-daf4-4817-9b6e-f017d971fc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:34.231564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8588-daf4-4817-9b6e-f017d971fc71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:34.232288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d8588-daf4-4817-9b6e-f017d971fc71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:34.232865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d8588-daf4-4817-9b6e-f017d971fc71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:34.240087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d8588-daf4-4817-9b6e-f017d971fc71 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:34.256673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:34.258584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:34.261610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:34.261610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:34.261610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:34.277245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:34.279363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 871/1771] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:34.281868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfe466-e246-4ab3-ab03-cb6c0ccdab18 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:34.285287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfe466-e246-4ab3-ab03-cb6c0ccdab18 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:34.285780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfe466-e246-4ab3-ab03-cb6c0ccdab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:34.322373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 901/1772] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:34.326911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:34.354352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493e8b9-be41-4b8a-a5b2-bee78cbb4e25 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:34.365748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2bdfe466-e246-4ab3-ab03-cb6c0ccdab18 None None] MFA Rules not processed for user `3835260bf10c4ab7b4f1b1ebfee9b1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:34.367938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:34.368595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:34.368888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:34.369170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:34.369864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:34.371176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2bdfe466-e246-4ab3-ab03-cb6c0ccdab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:34.371883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:34.530305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 902/1773] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:34 2022] POST /identity/v3/projects => generated 365 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:34.531849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:34.695521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 872/1774] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:34.698261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfe466-e246-4ab3-ab03-cb6c0ccdab18 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:34.719214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfe466-e246-4ab3-ab03-cb6c0ccdab18 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:34.724537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:34.725341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:34.725729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:34.726108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:34.726924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:34.729107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:34.739001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87af4c-a65d-4d34-803b-c4997003eaeb admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:34.760771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:34.761665 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:34.761811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:34.766761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:34.769659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:34.771337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:34.800171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 903/1775] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:34.801152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:34.808813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:34.891192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 873/1776] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:34 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:34.894895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:34.907927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:34.991787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c3906-af61-4d1f-9319-46cc5bc23a21 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:35.001483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:35.002862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.003566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.003802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.058883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b1495-f17c-45da-a112-c48a8dbac252 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:35.069588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:35.070675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.071167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.072118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.078044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:35.116937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:35.216426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:35.220467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:35.247886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 904/1777] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:00:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:00:35.250358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:35.252098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:35.256897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:35.279715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09feb7f-03f6-4dfc-a726-2b37fa19c683 service swift] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:35.286451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 874/1778] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:35.287411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:35.292451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:35.293091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.296176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.296490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] PATH_INFO: `/v3/projects/ac9fc94b0d6e4c8d888716d39583dc39/users/a040a34d70e449e1b519e43fc6695668/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.315964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e0da-1d82-4f22-aa07-ac2fc91feba3 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:35.328812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59882c5-a24a-4e0e-af35-25cc70be305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:35.329489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59882c5-a24a-4e0e-af35-25cc70be305b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.330538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59882c5-a24a-4e0e-af35-25cc70be305b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.330832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59882c5-a24a-4e0e-af35-25cc70be305b admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.333312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59882c5-a24a-4e0e-af35-25cc70be305b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:35.337880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59882c5-a24a-4e0e-af35-25cc70be305b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:35.340380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9fc94b0d6e4c8d888716d39583dc39, user_id=a040a34d70e449e1b519e43fc6695668, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:35.346156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:35.383126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 875/1779] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:35.387944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56672-d3ea-4e39-b322-9796ef93a538 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.387944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56672-d3ea-4e39-b322-9796ef93a538 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.387944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56672-d3ea-4e39-b322-9796ef93a538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.448229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 905/1780] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:35 2022] PUT /identity/v3/projects/ac9fc94b0d6e4c8d888716d39583dc39/users/a040a34d70e449e1b519e43fc6695668/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:35.449034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:35.480448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-cce56672-d3ea-4e39-b322-9796ef93a538 None None] MFA Rules not processed for user `ad3cf8a3e0a346fa867445d6b6300447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:35.485220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cce56672-d3ea-4e39-b322-9796ef93a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:35.686048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe670a67-5139-4f5e-bce2-af673be4ece3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:35.693525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 876/1781] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Aug 1 08:00:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:35.698446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.698446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.698771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.714891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:35.714891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.715810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.716064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.716760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:35.723010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:35.766646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 906/1782] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:35.770226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:35.771816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:35.773091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:35.811788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8861a32-a43a-4587-beed-a5367eef1b2c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:35.812197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 877/1783] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:35.817388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:35.821081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:35.840916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380f2d-33e2-4317-a66b-158471ca52d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:35.844753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f2d-33e2-4317-a66b-158471ca52d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.845057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f2d-33e2-4317-a66b-158471ca52d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.846714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f2d-33e2-4317-a66b-158471ca52d6 admin admin] PATH_INFO: `/v3/projects/aac84faad10a425cb0973248fa62d56c/users/40a13fef2c84438faeadcd8304f26305/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.872910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daaa28-0283-4eae-87ef-da59068668ea None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:35.888788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:35.890117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:35.890117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:35.890117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:35.916240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380f2d-33e2-4317-a66b-158471ca52d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac84faad10a425cb0973248fa62d56c, user_id=40a13fef2c84438faeadcd8304f26305, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:35.931598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380f2d-33e2-4317-a66b-158471ca52d6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:35.945852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:35.950198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:35.975105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 878/1784] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:35.976425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:36.000029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346400b-5ac6-4743-be0e-52a8e9917fac service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:36.022077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:36.022732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:36.023331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:36.023459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:36.042243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:36.045047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:36.081421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 907/1785] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:35 2022] PUT /identity/v3/projects/aac84faad10a425cb0973248fa62d56c/users/40a13fef2c84438faeadcd8304f26305/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:36.083098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70d798-0cec-4445-abfd-191dffcc6515 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:36.083384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70d798-0cec-4445-abfd-191dffcc6515 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:36.083695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70d798-0cec-4445-abfd-191dffcc6515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:36.169418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 879/1786] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:36.201685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cf70d798-0cec-4445-abfd-191dffcc6515 None None] MFA Rules not processed for user `40a13fef2c84438faeadcd8304f26305`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:36.203789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cf70d798-0cec-4445-abfd-191dffcc6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:36.405519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 908/1787] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:36.410563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:36.426301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ec052-5731-4fa6-9b48-306289fd858e service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:36.657551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:36.658543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:36.658838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:36.659804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:36.660007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:36.663429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:36.806406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 880/1788] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:36 2022] POST /identity/v3/projects => generated 363 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:36.806827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70d798-0cec-4445-abfd-191dffcc6515 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:36.824276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70d798-0cec-4445-abfd-191dffcc6515 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:37.023040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:37.028838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:37.028838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:37.028838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:37.028838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:37.028838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:37.410997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:37.423910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 909/1789] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:36 2022] POST /identity/v3/users => generated 288 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:00:37.426891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:37.449218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b66dbf-d7d7-4c67-9008-6f2af0a32983 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:37.467735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:37.470109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:37.472299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:37.472733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:37.474567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:37.479896 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:37.510529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 881/1790] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:37.515945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:37.536305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba177c-ea21-4ac6-9ee7-e8fa7d5d14e9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:37.548761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:37.555177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:37.555177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:37.555177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] PATH_INFO: `/v3/projects/82f321a5a07a477997c50bad64dbc8d8/users/36f81896a5f44df39ee82c5131c20f28/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:37.592302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f321a5a07a477997c50bad64dbc8d8, user_id=36f81896a5f44df39ee82c5131c20f28, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:37.597709 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:37.741902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 910/1791] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:37 2022] PUT /identity/v3/projects/82f321a5a07a477997c50bad64dbc8d8/users/36f81896a5f44df39ee82c5131c20f28/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:37.743791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:37.780633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6840169-8ef5-431b-a3ba-b13dc59a5344 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:37.871179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fa086-73e2-4e3b-a449-083c968d3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:37.873433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fa086-73e2-4e3b-a449-083c968d3397 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:37.873433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fa086-73e2-4e3b-a449-083c968d3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:37.873433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fa086-73e2-4e3b-a449-083c968d3397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:37.874042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fa086-73e2-4e3b-a449-083c968d3397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:37.875907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fa086-73e2-4e3b-a449-083c968d3397 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:37.921518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 882/1792] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:37.928362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:37.947437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338a04f-27bb-42b4-ad16-7accc26bb5f1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:37.964434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:37.965579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:37.966161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:37.966539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] PATH_INFO: `/v3/projects/82f321a5a07a477997c50bad64dbc8d8/users/36f81896a5f44df39ee82c5131c20f28/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:37.971745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f321a5a07a477997c50bad64dbc8d8, user_id=36f81896a5f44df39ee82c5131c20f28, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:37.982824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:38.012876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 911/1793] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:37 2022] PUT /identity/v3/projects/82f321a5a07a477997c50bad64dbc8d8/users/36f81896a5f44df39ee82c5131c20f28/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:38.025059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:38.025881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:38.025881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:38.088058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] MFA Rules not processed for user `36f81896a5f44df39ee82c5131c20f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:38.093991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:38.324556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 883/1794] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:38.334215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:38.492316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbd300-e21d-4553-917e-4023cbd71d0b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:38.501596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540ade2-6d11-4a51-8aae-f48caeaece8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:38.501596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540ade2-6d11-4a51-8aae-f48caeaece8b service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:38.501596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540ade2-6d11-4a51-8aae-f48caeaece8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:38.501596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540ade2-6d11-4a51-8aae-f48caeaece8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:38.513312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540ade2-6d11-4a51-8aae-f48caeaece8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:38.516403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540ade2-6d11-4a51-8aae-f48caeaece8b service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:38.532481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 912/1795] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:38.748923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:38.968025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:38.970089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:39.002797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974363a-e39a-4e3c-864d-0104249e7e03 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:39.014620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:39.015749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:39.020481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:39.020481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:39.257958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:39.260106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:39.274532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 884/1796] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:39.480662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:39.513120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5333e5-dab7-4941-bd0a-b84e045abee5 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:39.513572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5333e5-dab7-4941-bd0a-b84e045abee5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:39.513935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5333e5-dab7-4941-bd0a-b84e045abee5 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:39.517294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 913/1797] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:00:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:39.641329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:39.651986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da6c39-8987-44a7-9091-3bbdc3b73076 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:39.657651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:39.659341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:39.659842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:39.660221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:39.662598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:39.745284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c191c2-0901-4d68-8ac1-5d42d25c0b4c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:39.745574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c191c2-0901-4d68-8ac1-5d42d25c0b4c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:39.745849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c191c2-0901-4d68-8ac1-5d42d25c0b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:39.800146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-57c191c2-0901-4d68-8ac1-5d42d25c0b4c None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:39.801688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-57c191c2-0901-4d68-8ac1-5d42d25c0b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:39.820763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:39.822240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:39.844390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 885/1798] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:39.962769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 914/1799] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:00:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:40.536291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:40.537959 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:40.569462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0fb4b-6854-4559-85f1-32324a3a62a4 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:40.585090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402df0c0-d482-45b3-84a5-08db65249077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:40.586086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df0c0-d482-45b3-84a5-08db65249077 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:40.586502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df0c0-d482-45b3-84a5-08db65249077 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:40.586872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df0c0-d482-45b3-84a5-08db65249077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:40.606130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402df0c0-d482-45b3-84a5-08db65249077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:40.609158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402df0c0-d482-45b3-84a5-08db65249077 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:40.633325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 886/1800] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:40.706895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:40.909081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:41.979979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68f6f-90a5-46b5-a1cb-cc7cd9aa1777 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:41.981961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68f6f-90a5-46b5-a1cb-cc7cd9aa1777 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:41.982378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68f6f-90a5-46b5-a1cb-cc7cd9aa1777 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:41.984704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 915/1801] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:00:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:00:42.304115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:42.304115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:42.304115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:42.331616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68f6f-90a5-46b5-a1cb-cc7cd9aa1777 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:42.352780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:42.355104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:42.356108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:42.358483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68f6f-90a5-46b5-a1cb-cc7cd9aa1777 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:42.375996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1845f-5661-4b8c-ac9d-b5cd177cc65b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:42.380769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 887/1802] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:00:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:00:42.381876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1845f-5661-4b8c-ac9d-b5cd177cc65b service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:42.387445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1845f-5661-4b8c-ac9d-b5cd177cc65b service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:42.387445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1845f-5661-4b8c-ac9d-b5cd177cc65b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:42.405986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:42.564406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1845f-5661-4b8c-ac9d-b5cd177cc65b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:42.565987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1845f-5661-4b8c-ac9d-b5cd177cc65b service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:42.586191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 916/1803] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:43.151180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:43.153240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:43.181493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad26bd-d027-4ff9-8fc1-8781ba2797fb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:43.189990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:43.190936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:43.193389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:43.193389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:43.228329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:43.230155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:43.258819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 888/1804] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:43.817120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:43.892735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099a6b1-2267-466e-8b77-9de8092f69c4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:43.893837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099a6b1-2267-466e-8b77-9de8092f69c4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:43.894139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099a6b1-2267-466e-8b77-9de8092f69c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:43.970645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5099a6b1-2267-466e-8b77-9de8092f69c4 None None] MFA Rules not processed for user `a040a34d70e449e1b519e43fc6695668`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:43.972169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5099a6b1-2267-466e-8b77-9de8092f69c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:44.143220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 917/1805] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:44.148364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:44.281015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8578-773b-4477-8c25-635d99f077f8 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:44.286836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:44.288632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:44.289003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:44.289840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:44.290573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:44.295152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:44.312134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099a6b1-2267-466e-8b77-9de8092f69c4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:44.327263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099a6b1-2267-466e-8b77-9de8092f69c4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:44.333018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:44.333688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:44.333968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:44.337516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] PATH_INFO: `/v3/users/40a13fef2c84438faeadcd8304f26305` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:44.337516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a13fef2c84438faeadcd8304f26305)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:44.338216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:44.414109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 889/1806] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:44 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:44.417563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:44.553065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a13fef2c84438faeadcd8304f26305'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:44.610253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66021704-980b-4a75-aacb-37f539a02f59 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:44.616430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:44.617955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:44.618266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:44.618546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:44.621731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:44.624422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:44.674570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a13fef2c84438faeadcd8304f26305'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:44.686985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a13fef2c84438faeadcd8304f26305'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:44.687888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a13fef2c84438faeadcd8304f26305'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:44.691571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1c322a9e-05bb-4206-8705-0969457f4698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a13fef2c84438faeadcd8304f26305'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:44.717552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 918/1807] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:44 2022] DELETE /identity/v3/users/40a13fef2c84438faeadcd8304f26305 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:44.720712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:44.720712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:44.720902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:44.750500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:44.756652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:44.771037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 890/1808] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:44 2022] POST /identity/v3/users => generated 293 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:44.773203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:44.781758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:44.844908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b0b3f-a2f3-4a46-8ce3-e6f81a37d841 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:44.928682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 919/1809] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:44.960621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:44.970937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:44.973184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:44.974808 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:44.994544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:44.994544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:44.994544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:44.994544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:44.994544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:45.007470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 891/1810] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:45.014511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:45.033580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870794b-8d95-44e6-887e-ebeaf4849ee3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:45.043161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:45.044987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:45.045422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:45.045857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] PATH_INFO: `/v3/projects/064a8d25e8164a4e9f5439a3d8613609/users/0edfdfcb1a6a4b0baf8a70c9e3fb698a/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:45.071867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4671-bcad-47b6-aa35-14ba7227c5da None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:45.076528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064a8d25e8164a4e9f5439a3d8613609, user_id=0edfdfcb1a6a4b0baf8a70c9e3fb698a, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:45.077066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:45.077718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:45.078418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:45.079278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:45.090902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:45.092892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:45.094946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:45.111905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 920/1811] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:45.185765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 892/1812] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:45 2022] PUT /identity/v3/projects/064a8d25e8164a4e9f5439a3d8613609/users/0edfdfcb1a6a4b0baf8a70c9e3fb698a/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:45.190124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:45.224576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3545d79-62da-4ac2-b6f5-0691024fc23a service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:45.332844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:45.334203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:45.334727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:45.335123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:45.337091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:45.339327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:45.357550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 921/1813] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:45.360835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:45.384431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf54241-e714-404c-b147-45480f5f4406 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:45.398998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:45.400045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:45.402481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:45.404052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] PATH_INFO: `/v3/projects/064a8d25e8164a4e9f5439a3d8613609/users/0edfdfcb1a6a4b0baf8a70c9e3fb698a/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:45.410639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064a8d25e8164a4e9f5439a3d8613609, user_id=0edfdfcb1a6a4b0baf8a70c9e3fb698a, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:45.413608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:45.443779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 893/1814] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:45 2022] PUT /identity/v3/projects/064a8d25e8164a4e9f5439a3d8613609/users/0edfdfcb1a6a4b0baf8a70c9e3fb698a/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:45.754247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:45.918489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:45.921145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:45.946914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae4438-b9df-4d4e-bf5b-b2838be66517 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:46.034333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:46.034976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:46.035259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:46.035584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] PATH_INFO: `/v3/projects/aac84faad10a425cb0973248fa62d56c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:46.036732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac84faad10a425cb0973248fa62d56c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:46.038637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:46.135725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac84faad10a425cb0973248fa62d56c', 'request_id': 'req-431b292f-7b53-49b4-bc27-6a0e8a43242e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.238512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac84faad10a425cb0973248fa62d56c', 'request_id': 'req-431b292f-7b53-49b4-bc27-6a0e8a43242e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.245518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac84faad10a425cb0973248fa62d56c', 'request_id': 'req-431b292f-7b53-49b4-bc27-6a0e8a43242e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.258323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] The token cache is being invalidate because project aac84faad10a425cb0973248fa62d56c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:46.258538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.259191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-431b292f-7b53-49b4-bc27-6a0e8a43242e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.260819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 922/1815] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:45 2022] DELETE /identity/v3/projects/aac84faad10a425cb0973248fa62d56c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:46.265577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:46.267286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:46.280314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a3d0d-d05d-4e00-875b-93033432ff05 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:46.339092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:46.339830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:46.340115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:46.340423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] PATH_INFO: `/v3/users/36f81896a5f44df39ee82c5131c20f28` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:46.341623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36f81896a5f44df39ee82c5131c20f28)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:46.343673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:46.552146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36f81896a5f44df39ee82c5131c20f28'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.559973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36f81896a5f44df39ee82c5131c20f28'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.633727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36f81896a5f44df39ee82c5131c20f28'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.634380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36f81896a5f44df39ee82c5131c20f28'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.646723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36f81896a5f44df39ee82c5131c20f28'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:46.653123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 894/1816] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:46 2022] DELETE /identity/v3/users/36f81896a5f44df39ee82c5131c20f28 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:46.996780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38213a88-90e9-4560-86dc-633db062398b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:46.997270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38213a88-90e9-4560-86dc-633db062398b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:46.997364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38213a88-90e9-4560-86dc-633db062398b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:47.028658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-38213a88-90e9-4560-86dc-633db062398b None None] MFA Rules not processed for user `0edfdfcb1a6a4b0baf8a70c9e3fb698a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:47.030399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-38213a88-90e9-4560-86dc-633db062398b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:47.045251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:47.140131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 923/1817] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:47.153105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:47.154516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:47.213498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bc68b-c0e1-4d4a-af3f-eaac81028985 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:47.217876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c18e5-fdc1-4aad-bb6b-0021f6ac91db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:47.218511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c18e5-fdc1-4aad-bb6b-0021f6ac91db service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:47.218800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c18e5-fdc1-4aad-bb6b-0021f6ac91db service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:47.219078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c18e5-fdc1-4aad-bb6b-0021f6ac91db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:47.221037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-101c18e5-fdc1-4aad-bb6b-0021f6ac91db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:47.280194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c18e5-fdc1-4aad-bb6b-0021f6ac91db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:47.281637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c18e5-fdc1-4aad-bb6b-0021f6ac91db service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:47.290443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 895/1818] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:47.303748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38213a88-90e9-4560-86dc-633db062398b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:47.305151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-38213a88-90e9-4560-86dc-633db062398b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:47.428096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38213a88-90e9-4560-86dc-633db062398b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:47.437540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:47.438432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:47.439259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:47.439833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] PATH_INFO: `/v3/projects/82f321a5a07a477997c50bad64dbc8d8` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:47.441502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f321a5a07a477997c50bad64dbc8d8)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:47.445119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:47.616208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f321a5a07a477997c50bad64dbc8d8', 'request_id': 'req-3a4a0145-ee01-461f-9383-1d233ab632f7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:47.691713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:47.719734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f321a5a07a477997c50bad64dbc8d8', 'request_id': 'req-3a4a0145-ee01-461f-9383-1d233ab632f7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:47.728702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f321a5a07a477997c50bad64dbc8d8', 'request_id': 'req-3a4a0145-ee01-461f-9383-1d233ab632f7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:47.739148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] The token cache is being invalidate because project 82f321a5a07a477997c50bad64dbc8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:47.739846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:47.741096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3a4a0145-ee01-461f-9383-1d233ab632f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:47.744292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 924/1819] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:47 2022] DELETE /identity/v3/projects/82f321a5a07a477997c50bad64dbc8d8 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:47.756890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63e6b2-5a2a-4948-843c-34d24931e76d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:47.756890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63e6b2-5a2a-4948-843c-34d24931e76d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:47.756890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63e6b2-5a2a-4948-843c-34d24931e76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:47.797086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6f63e6b2-5a2a-4948-843c-34d24931e76d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:47.798753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6f63e6b2-5a2a-4948-843c-34d24931e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:47.895296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 896/1820] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:47.897181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2f80b-add3-4b40-8c1a-d2fca669113b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:47.897591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2f80b-add3-4b40-8c1a-d2fca669113b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:47.898353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2f80b-add3-4b40-8c1a-d2fca669113b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:47.922594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9ef2f80b-add3-4b40-8c1a-d2fca669113b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:47.923884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9ef2f80b-add3-4b40-8c1a-d2fca669113b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:47.932476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 925/1821] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:47.937777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63e6b2-5a2a-4948-843c-34d24931e76d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:47.949783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63e6b2-5a2a-4948-843c-34d24931e76d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:47.954540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:47.955174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:47.955462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:47.955774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:47.957651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:47.957651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:47.958809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:47.973170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 897/1822] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:00:47.978056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2f80b-add3-4b40-8c1a-d2fca669113b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:47.988393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2f80b-add3-4b40-8c1a-d2fca669113b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:47.993453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:47.994081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:47.994395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:47.994677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:47.995346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:47.997889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:48.128355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 926/1823] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:47 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:00:48.132066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:48.143847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94021f2b-1ec3-41c0-8358-e1a94885dc58 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:48.223318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:48.224015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:48.224326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:48.224603 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:48.225277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:48.233105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:48.330807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 898/1824] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:48 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:48.332745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:48.344217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944e55-a6b1-4d73-8f71-337d85f2d398 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:48.348891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:48.349662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:48.349951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:48.351155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:48.351155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:48.352580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:48.366481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 927/1825] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:00:48.370128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:48.383116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b9050-d35c-4888-a1a2-cd0b773d19cb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:48.389214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f6d04-d832-4b21-81bf-d3895778836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:48.389214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f6d04-d832-4b21-81bf-d3895778836a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:48.389214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f6d04-d832-4b21-81bf-d3895778836a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:48.389578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f6d04-d832-4b21-81bf-d3895778836a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:48.390287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f6d04-d832-4b21-81bf-d3895778836a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:48.392027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f6d04-d832-4b21-81bf-d3895778836a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:48.402976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 899/1826] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:48.407255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:48.419185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea6b31-dcb4-4d63-84e8-a69867fa098d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:48.431694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:48.432580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:48.433037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:48.434052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] PATH_INFO: `/v3/projects/37c7a50c73ca4c649d5f902247580a56/users/2eb3ea08c1094a02a98594e0ae34d19a/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:48.456200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c7a50c73ca4c649d5f902247580a56, user_id=2eb3ea08c1094a02a98594e0ae34d19a, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:48.459055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:48.581324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:48.598143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 928/1827] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:48 2022] PUT /identity/v3/projects/37c7a50c73ca4c649d5f902247580a56/users/2eb3ea08c1094a02a98594e0ae34d19a/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:48.611515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf515c-385b-4c4b-a3ae-337aa93dedab None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:48.611831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf515c-385b-4c4b-a3ae-337aa93dedab None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:48.612112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf515c-385b-4c4b-a3ae-337aa93dedab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:48.642732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0daf515c-385b-4c4b-a3ae-337aa93dedab None None] MFA Rules not processed for user `2eb3ea08c1094a02a98594e0ae34d19a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:48.644461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0daf515c-385b-4c4b-a3ae-337aa93dedab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:48.725120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 900/1828] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:48.729699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:48.869033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2863-211b-4321-8814-b13834a4e029 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:48.869033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:48.869033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:48.869033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:48.869322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:48.869322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:48.869322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:48.983564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 929/1829] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:48 2022] POST /identity/v3/projects => generated 387 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:00:48.986103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf515c-385b-4c4b-a3ae-337aa93dedab None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:48.998103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf515c-385b-4c4b-a3ae-337aa93dedab None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:49.078939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:49.079752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.080065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.080361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.081028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:49.083110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:49.189148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 901/1830] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:48 2022] POST /identity/v3/users => generated 300 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:49.193216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:49.205135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae50f3-474d-43bc-b43b-0d527eb1285e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:49.210220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:49.210952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.211336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.211773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.212582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:49.214771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:49.230874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 930/1831] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:49.234020 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:49.244927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a404c-797f-4b64-8b91-60a0be1a2b87 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:49.250655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:49.251333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.251648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.251928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] PATH_INFO: `/v3/projects/28e98cf1101e48eabb7b15aa54a47d26/users/c969aab2bb554141bffc4568fcafac39/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.268248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e98cf1101e48eabb7b15aa54a47d26, user_id=c969aab2bb554141bffc4568fcafac39, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:49.270906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:49.279482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:49.359637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 902/1832] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:49 2022] PUT /identity/v3/projects/28e98cf1101e48eabb7b15aa54a47d26/users/c969aab2bb554141bffc4568fcafac39/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:49.363745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:49.374819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858eb14d-b764-44fd-88b3-cab761c3fb5b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:49.434472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace545b-83d8-4bb2-b364-8cdd439ab42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:49.435125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace545b-83d8-4bb2-b364-8cdd439ab42b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.435409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace545b-83d8-4bb2-b364-8cdd439ab42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.435752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace545b-83d8-4bb2-b364-8cdd439ab42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.436494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace545b-83d8-4bb2-b364-8cdd439ab42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:49.438335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace545b-83d8-4bb2-b364-8cdd439ab42b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:49.439779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:49.450157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 931/1833] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:49.454145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:49.465259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c68fa-cad3-4238-a790-d520f2657435 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:49.470173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:49.472986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.472986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.472986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] PATH_INFO: `/v3/projects/28e98cf1101e48eabb7b15aa54a47d26/users/c969aab2bb554141bffc4568fcafac39/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.473844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e98cf1101e48eabb7b15aa54a47d26, user_id=c969aab2bb554141bffc4568fcafac39, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:49.476525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:49.489432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 903/1834] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:49 2022] PUT /identity/v3/projects/28e98cf1101e48eabb7b15aa54a47d26/users/c969aab2bb554141bffc4568fcafac39/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:49.500273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc8536-db60-4982-8791-461a40095d9d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.500595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc8536-db60-4982-8791-461a40095d9d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.500875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc8536-db60-4982-8791-461a40095d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.531445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a4dc8536-db60-4982-8791-461a40095d9d None None] MFA Rules not processed for user `c969aab2bb554141bffc4568fcafac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:49.535958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a4dc8536-db60-4982-8791-461a40095d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:49.596159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:49.639362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 932/1835] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:49.639799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:49.650599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:49.652741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:49.715489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3fe5-4e5c-41c0-a408-b4eb0b1fac70 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:49.719742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0695c4-2a26-416b-9c61-740a698c178d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:49.723347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0695c4-2a26-416b-9c61-740a698c178d service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.723347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0695c4-2a26-416b-9c61-740a698c178d service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.723347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0695c4-2a26-416b-9c61-740a698c178d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.732103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0695c4-2a26-416b-9c61-740a698c178d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:49.733556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0695c4-2a26-416b-9c61-740a698c178d service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:49.743502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 904/1836] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:49.820669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc8536-db60-4982-8791-461a40095d9d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:49.887162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc8536-db60-4982-8791-461a40095d9d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:49.892216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:49.892904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:49.895832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:49.895832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] PATH_INFO: `/v3/users/2eb3ea08c1094a02a98594e0ae34d19a` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:49.895832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb3ea08c1094a02a98594e0ae34d19a)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:49.896818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:50.028654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:50.116360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb3ea08c1094a02a98594e0ae34d19a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:50.194644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb3ea08c1094a02a98594e0ae34d19a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:50.203837 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb3ea08c1094a02a98594e0ae34d19a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:50.204817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb3ea08c1094a02a98594e0ae34d19a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:50.205583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb3ea08c1094a02a98594e0ae34d19a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:50.236476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 933/1837] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:49 2022] DELETE /identity/v3/users/2eb3ea08c1094a02a98594e0ae34d19a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:50.248795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:50.249089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:50.249374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:50.279311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:50.279311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:50.308579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:50.358510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 905/1838] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:50.368938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:50.370868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:50.438836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaf6f3-a227-416d-b6eb-4314975b5a75 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:50.442653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:50.443320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:50.443643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:50.443923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:50.454980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:50.456537 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:50.467861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 934/1839] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:51.303515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:51.305186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:51.319277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986ba1-5a32-4539-8b29-262642a8dad0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:51.324465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:51.325163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:51.325452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:51.326668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] PATH_INFO: `/v3/projects/37c7a50c73ca4c649d5f902247580a56` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:51.328092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c7a50c73ca4c649d5f902247580a56)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:51.329925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:51.384251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:51.473493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c7a50c73ca4c649d5f902247580a56', 'request_id': 'req-7686c0fa-d916-466f-b558-a3a188ab43b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.480741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c7a50c73ca4c649d5f902247580a56', 'request_id': 'req-7686c0fa-d916-466f-b558-a3a188ab43b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.486472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c7a50c73ca4c649d5f902247580a56', 'request_id': 'req-7686c0fa-d916-466f-b558-a3a188ab43b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.576167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] The token cache is being invalidate because project 37c7a50c73ca4c649d5f902247580a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:51.576464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.577146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.578695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 906/1840] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:51 2022] DELETE /identity/v3/projects/37c7a50c73ca4c649d5f902247580a56 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:51.582707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:51.593237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e169e-86a3-4307-888d-d3111daae6a4 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:51.649256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:51.649902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:51.650222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:51.650505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] PATH_INFO: `/v3/users/c969aab2bb554141bffc4568fcafac39` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:51.652230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c969aab2bb554141bffc4568fcafac39)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:51.656121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:51.695053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:51.824701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c969aab2bb554141bffc4568fcafac39'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.918809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c969aab2bb554141bffc4568fcafac39'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.929739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c969aab2bb554141bffc4568fcafac39'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.930512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c969aab2bb554141bffc4568fcafac39'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.931221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c969aab2bb554141bffc4568fcafac39'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:51.955482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 935/1841] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:51 2022] DELETE /identity/v3/users/c969aab2bb554141bffc4568fcafac39 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:52.224443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:52.473429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:52.473429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:52.535594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686c0fa-d916-466f-b558-a3a188ab43b8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:52.589988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f3610-3dc0-4638-9d69-ade9540df99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:52.590687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f3610-3dc0-4638-9d69-ade9540df99c service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:52.590984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f3610-3dc0-4638-9d69-ade9540df99c service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:52.591248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f3610-3dc0-4638-9d69-ade9540df99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:52.593292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-061f3610-3dc0-4638-9d69-ade9540df99c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:52.662277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f3610-3dc0-4638-9d69-ade9540df99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:52.663690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f3610-3dc0-4638-9d69-ade9540df99c service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:52.675637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 907/1842] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:52.870578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:52.873889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:52.904442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:52.956171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4787f0-1ec3-40ca-b838-2b70c52e89a3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:52.963417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:52.965630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:52.966118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:52.966905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] PATH_INFO: `/v3/projects/28e98cf1101e48eabb7b15aa54a47d26` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:52.968585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e98cf1101e48eabb7b15aa54a47d26)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:52.970992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:53.090960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e98cf1101e48eabb7b15aa54a47d26', 'request_id': 'req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:53.147552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e98cf1101e48eabb7b15aa54a47d26', 'request_id': 'req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:53.154917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e98cf1101e48eabb7b15aa54a47d26', 'request_id': 'req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:53.176969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] The token cache is being invalidate because project 28e98cf1101e48eabb7b15aa54a47d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:53.176969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:53.176969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8cdadf0e-fedd-4c5e-88ae-3f1b7604a1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:53.176969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 936/1843] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:52 2022] DELETE /identity/v3/projects/28e98cf1101e48eabb7b15aa54a47d26 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:53.180740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c04fc-ec52-48d3-910d-0a9b705b4011 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.181188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c04fc-ec52-48d3-910d-0a9b705b4011 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.181476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c04fc-ec52-48d3-910d-0a9b705b4011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.204978 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1c8c04fc-ec52-48d3-910d-0a9b705b4011 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:53.206382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1c8c04fc-ec52-48d3-910d-0a9b705b4011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:53.242260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:53.282797 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 908/1844] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:53.295034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecb681-08e7-4419-b776-54b624efc3cf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.295690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecb681-08e7-4419-b776-54b624efc3cf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.296170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecb681-08e7-4419-b776-54b624efc3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.325997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-63ecb681-08e7-4419-b776-54b624efc3cf None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:53.327696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-63ecb681-08e7-4419-b776-54b624efc3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:53.337187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 937/1845] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:53.342182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c04fc-ec52-48d3-910d-0a9b705b4011 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:53.352560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c04fc-ec52-48d3-910d-0a9b705b4011 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:53.358697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:53.359339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.359833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.360124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.361261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:53.361799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:53.367784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:53.377507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 909/1846] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:00:53.388859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecb681-08e7-4419-b776-54b624efc3cf None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:53.396198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecb681-08e7-4419-b776-54b624efc3cf None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:53.401233 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:53.402764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.403149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.403423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.404177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:53.406079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:53.493019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 938/1847] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:53 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:53.497320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:53.508144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450d4c5-90dc-467c-b426-eb8428f9ad44 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:53.569490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:53.570125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.570411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.570688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.571356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:53.574064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:53.662435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 910/1848] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:53 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:53.666637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:53.677306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f155a8b-f9d6-4e3e-8a12-b92d3ded88f4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:53.682299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:53.683724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.684047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.684333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.685070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:53.686732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:53.701430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 939/1849] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:53.701866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:53.712488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b3c3b-4b9d-46a2-a6ac-7732bb1c8598 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:53.717506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f20a61-e676-442a-b63a-b315a9dcd567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:53.718133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f20a61-e676-442a-b63a-b315a9dcd567 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.718417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f20a61-e676-442a-b63a-b315a9dcd567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.718692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f20a61-e676-442a-b63a-b315a9dcd567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.719401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f20a61-e676-442a-b63a-b315a9dcd567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:53.721076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f20a61-e676-442a-b63a-b315a9dcd567 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:53.732223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 911/1850] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:00:53.736479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:53.747415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30e9d4-7098-4052-a132-8ce2dfe0d14f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:53.753808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:53.753808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.753808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.753808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] PATH_INFO: `/v3/projects/ea939703b48b49488429cbe24fe8be21/users/8bbd9612c828464b8475ae6bb845354d/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.769956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea939703b48b49488429cbe24fe8be21, user_id=8bbd9612c828464b8475ae6bb845354d, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:53.772827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:53.885063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 940/1851] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:53 2022] PUT /identity/v3/projects/ea939703b48b49488429cbe24fe8be21/users/8bbd9612c828464b8475ae6bb845354d/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:53.896349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b091fab-2d83-4ac8-8678-545c95cfde14 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:53.896720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b091fab-2d83-4ac8-8678-545c95cfde14 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:53.897097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b091fab-2d83-4ac8-8678-545c95cfde14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:53.926218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4b091fab-2d83-4ac8-8678-545c95cfde14 None None] MFA Rules not processed for user `8bbd9612c828464b8475ae6bb845354d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:53.927468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4b091fab-2d83-4ac8-8678-545c95cfde14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:53.999494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 912/1852] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:54.009635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:54.012525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:54.016501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:54.018002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:54.069898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7dfb9-6d50-4eee-9ca3-1327e9fb7a2d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:54.073534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d39f43-b07a-47ca-b8c7-6ff182fab73c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:54.074158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d39f43-b07a-47ca-b8c7-6ff182fab73c service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:54.074447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d39f43-b07a-47ca-b8c7-6ff182fab73c service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:54.074727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d39f43-b07a-47ca-b8c7-6ff182fab73c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:54.082833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d39f43-b07a-47ca-b8c7-6ff182fab73c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:54.084328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d39f43-b07a-47ca-b8c7-6ff182fab73c service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:54.093240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 941/1853] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:54.115591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b091fab-2d83-4ac8-8678-545c95cfde14 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:54.177891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b091fab-2d83-4ac8-8678-545c95cfde14 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:54.182447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:54.183158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:54.183962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:54.184241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] PATH_INFO: `/v3/users/8bbd9612c828464b8475ae6bb845354d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:54.185716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bbd9612c828464b8475ae6bb845354d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:54.187657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:54.404545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bbd9612c828464b8475ae6bb845354d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:54.414640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bbd9612c828464b8475ae6bb845354d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:54.547785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bbd9612c828464b8475ae6bb845354d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:54.552778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bbd9612c828464b8475ae6bb845354d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:54.554844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:54.562892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bbd9612c828464b8475ae6bb845354d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:54.569282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 913/1854] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:54 2022] DELETE /identity/v3/users/8bbd9612c828464b8475ae6bb845354d => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:54.574703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:54.579265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:54.579265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:54.602469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:54.603725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:54.664108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 942/1855] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:54.674765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:54.676463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:54.735538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae7fad-02d1-41f5-a135-7c13d8d34212 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:54.739316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a58542-c88c-4091-8220-a0ea381d58e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:54.740066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a58542-c88c-4091-8220-a0ea381d58e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:54.740353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a58542-c88c-4091-8220-a0ea381d58e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:54.740917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a58542-c88c-4091-8220-a0ea381d58e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:54.750415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a58542-c88c-4091-8220-a0ea381d58e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:54.756783 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a58542-c88c-4091-8220-a0ea381d58e7 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:54.762488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 914/1856] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:00:55.211820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:55.213268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:55.225610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd1572-4661-41b1-ab19-04c5f3f666c6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:55.230945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:55.231715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.231999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.232269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] PATH_INFO: `/v3/projects/ea939703b48b49488429cbe24fe8be21` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.233438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea939703b48b49488429cbe24fe8be21)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:55.237368 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:55.325701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea939703b48b49488429cbe24fe8be21', 'request_id': 'req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:55.375842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea939703b48b49488429cbe24fe8be21', 'request_id': 'req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:55.381659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea939703b48b49488429cbe24fe8be21', 'request_id': 'req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:55.393891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] The token cache is being invalidate because project ea939703b48b49488429cbe24fe8be21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:55.394182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:55.394816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed5117dc-d8e6-4c6b-8220-dfc7e5a3c67f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:55.396371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 943/1857] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:55 2022] DELETE /identity/v3/projects/ea939703b48b49488429cbe24fe8be21 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:55.401705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f46ad-dc12-4c52-9e39-a16ca31f8795 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.401962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f46ad-dc12-4c52-9e39-a16ca31f8795 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.402241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f46ad-dc12-4c52-9e39-a16ca31f8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.423845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-194f46ad-dc12-4c52-9e39-a16ca31f8795 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:55.425047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-194f46ad-dc12-4c52-9e39-a16ca31f8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:55.483950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 915/1858] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:55.491658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70810ca3-846e-4bb4-b2f9-9ca9fae8a4fb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.491955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70810ca3-846e-4bb4-b2f9-9ca9fae8a4fb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.492242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70810ca3-846e-4bb4-b2f9-9ca9fae8a4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.513733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-70810ca3-846e-4bb4-b2f9-9ca9fae8a4fb None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:55.514933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-70810ca3-846e-4bb4-b2f9-9ca9fae8a4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:55.522782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 944/1859] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:55.527059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f46ad-dc12-4c52-9e39-a16ca31f8795 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:55.536653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f46ad-dc12-4c52-9e39-a16ca31f8795 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:55.541183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:55.542285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.542575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.542847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.543554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:00:55.543867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:55.545456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:55.557116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 916/1860] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:00:55.561938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70810ca3-846e-4bb4-b2f9-9ca9fae8a4fb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:55.571078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70810ca3-846e-4bb4-b2f9-9ca9fae8a4fb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:55.576723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:55.577330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.577639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.577913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.578577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:55.580511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:55.676169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:55.684661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 945/1861] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:00:55 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:55.689217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:55.700065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce80cb-831e-4989-942c-c80dbb4be39f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:55.773867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:55.775260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.775260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.775708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.776618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:55.778823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:55.897917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 917/1862] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:00:55 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:00:55.903946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:55.920372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd64f1a-b0a1-430c-9f3b-1344a5f096b6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:55.947846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:55.947846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.947846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.947846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.948737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:55.948737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:55.959645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 946/1863] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:00:55.965048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:55.976963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589102f-8356-46c4-9f43-72465a1701e8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:55.982699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fa8bb-d96d-4a43-a95d-67527e62c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:55.983412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fa8bb-d96d-4a43-a95d-67527e62c89d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:55.983791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fa8bb-d96d-4a43-a95d-67527e62c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:55.984136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fa8bb-d96d-4a43-a95d-67527e62c89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:55.985019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fa8bb-d96d-4a43-a95d-67527e62c89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:55.986698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fa8bb-d96d-4a43-a95d-67527e62c89d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:55.988476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:56.003591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 918/1864] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:00:56.006673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:56.019967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfecf13-dc0f-43f0-92d0-02e7fb6c5c5a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:56.032886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:56.037545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:56.037545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:56.037545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] PATH_INFO: `/v3/projects/f1a23acae56c4f67a6da88bd3af7b61c/users/06eb84f4363d4ffeaa2a6f317426b496/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:56.061688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a23acae56c4f67a6da88bd3af7b61c, user_id=06eb84f4363d4ffeaa2a6f317426b496, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:56.068688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:56.655218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 947/1865] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:00:56 2022] PUT /identity/v3/projects/f1a23acae56c4f67a6da88bd3af7b61c/users/06eb84f4363d4ffeaa2a6f317426b496/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:00:56.666246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:56.667221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:56.667320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:56.706643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] MFA Rules not processed for user `06eb84f4363d4ffeaa2a6f317426b496`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:56.707899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:56.779703 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 919/1866] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:56.791023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:56.793396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:56.878929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e341c-3292-4aa2-ba3a-d951408ad678 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:56.884479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8095fc-d122-4abe-8da2-cb3ad43cacc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:56.885107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8095fc-d122-4abe-8da2-cb3ad43cacc6 service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:56.885393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8095fc-d122-4abe-8da2-cb3ad43cacc6 service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:56.885675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8095fc-d122-4abe-8da2-cb3ad43cacc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:56.895012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8095fc-d122-4abe-8da2-cb3ad43cacc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:56.896600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8095fc-d122-4abe-8da2-cb3ad43cacc6 service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:56.906953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 948/1867] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:58.087939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:58.090761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:58.173224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1451592-3f27-4f7a-993a-258e3fad0d5a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:58.177922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:58.178575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:58.178862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:58.179136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] PATH_INFO: `/v3/users/87cb3e7f265940e19220664fe7c40b36` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:58.180357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87cb3e7f265940e19220664fe7c40b36)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:58.182912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:58.413812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87cb3e7f265940e19220664fe7c40b36'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:58.434259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87cb3e7f265940e19220664fe7c40b36'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:58.598772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87cb3e7f265940e19220664fe7c40b36'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:58.599674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87cb3e7f265940e19220664fe7c40b36'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:58.621530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87cb3e7f265940e19220664fe7c40b36'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:58.631317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 920/1868] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:58 2022] DELETE /identity/v3/users/87cb3e7f265940e19220664fe7c40b36 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:58.641360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:58.641360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:58.641360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:58.665726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:00:58.671626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:58.745035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 949/1869] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:00:58.754897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:58.756667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:58.768229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:58.820707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a81a-5ee4-4c21-9b24-248256da7a12 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:58.825027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:58.825683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:58.825966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:58.826241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:58.835663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:58.837284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:58.850015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 921/1870] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:00:59.291428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:59.297469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:00:59.306411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133d94c-4f98-41b2-bb83-35074372ef02 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:59.312639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:59.313297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:59.313589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:59.313859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] PATH_INFO: `/v3/projects/5fc90c87605841edb097fe3b12874712` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:59.315100 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc90c87605841edb097fe3b12874712)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:59.317937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:59.452215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:00:59.456282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc90c87605841edb097fe3b12874712', 'request_id': 'req-4cf9ed3d-3e29-4e39-8506-1e112e232897'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.551928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc90c87605841edb097fe3b12874712', 'request_id': 'req-4cf9ed3d-3e29-4e39-8506-1e112e232897'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.560417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc90c87605841edb097fe3b12874712', 'request_id': 'req-4cf9ed3d-3e29-4e39-8506-1e112e232897'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.573674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] The token cache is being invalidate because project 5fc90c87605841edb097fe3b12874712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:00:59.573971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.574660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.576474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 950/1871] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:00:59 2022] DELETE /identity/v3/projects/5fc90c87605841edb097fe3b12874712 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:00:59.580204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:00:59.590531 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c59f6-d517-4797-b1d8-b5ebad13397f service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:00:59.669316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:00:59.673645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:00:59.673645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:00:59.673645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] PATH_INFO: `/v3/users/a149c3b7c6f549dc92d42ba5019e59a1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:00:59.673645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a149c3b7c6f549dc92d42ba5019e59a1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:00:59.674285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:00:59.853771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a149c3b7c6f549dc92d42ba5019e59a1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.861631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a149c3b7c6f549dc92d42ba5019e59a1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.962140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a149c3b7c6f549dc92d42ba5019e59a1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.962878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a149c3b7c6f549dc92d42ba5019e59a1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.976806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a149c3b7c6f549dc92d42ba5019e59a1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:00:59.983381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 922/1872] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:00:59 2022] DELETE /identity/v3/users/a149c3b7c6f549dc92d42ba5019e59a1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:00.435035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:00.436788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:00.451468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9ed3d-3e29-4e39-8506-1e112e232897 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:00.510856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:00.512855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:00.513299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:00.513587 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] PATH_INFO: `/v3/projects/5c72a36a57094831aef34a02ba4c89e2` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:00.515508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c72a36a57094831aef34a02ba4c89e2)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:00.517502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:00.658699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c72a36a57094831aef34a02ba4c89e2', 'request_id': 'req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:00.730289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c72a36a57094831aef34a02ba4c89e2', 'request_id': 'req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:00.737945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c72a36a57094831aef34a02ba4c89e2', 'request_id': 'req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:00.755934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] The token cache is being invalidate because project 5c72a36a57094831aef34a02ba4c89e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:00.755934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:00.765254 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:00.769350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 951/1873] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:00 2022] DELETE /identity/v3/projects/5c72a36a57094831aef34a02ba4c89e2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:00.773596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:00.775375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:00.789843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25226aed-3d71-44d7-8bb5-82434dc4392f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:00.840358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:00.903866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:00.904764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:00.905200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:00.905635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] PATH_INFO: `/v3/users/3835260bf10c4ab7b4f1b1ebfee9b1f9` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:00.907094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3835260bf10c4ab7b4f1b1ebfee9b1f9)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:00.909148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:01.129533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3835260bf10c4ab7b4f1b1ebfee9b1f9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:01.136759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3835260bf10c4ab7b4f1b1ebfee9b1f9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:01.232178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3835260bf10c4ab7b4f1b1ebfee9b1f9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:01.232749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3835260bf10c4ab7b4f1b1ebfee9b1f9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:01.246955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3901c17e-d92d-4f1a-903d-845825c6851b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3835260bf10c4ab7b4f1b1ebfee9b1f9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:01.253710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 923/1874] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:00 2022] DELETE /identity/v3/users/3835260bf10c4ab7b4f1b1ebfee9b1f9 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:01.721961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:01.724491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:01.741377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbdcd7-b2f7-4d87-afa5-5afcd1852872 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:01.809006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:01.809584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:01.809872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:01.810152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] PATH_INFO: `/v3/projects/099e35d20cd94455988579feb25288c9` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:01.811337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=099e35d20cd94455988579feb25288c9)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:01.813422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:01.815862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:01.930920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099e35d20cd94455988579feb25288c9', 'request_id': 'req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:02.014702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099e35d20cd94455988579feb25288c9', 'request_id': 'req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:02.021766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099e35d20cd94455988579feb25288c9', 'request_id': 'req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:02.034095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] The token cache is being invalidate because project 099e35d20cd94455988579feb25288c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:02.034392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:02.035074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5a0033c-236f-4da0-8a16-53bdf9a0fb36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:02.036954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 952/1875] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:01 2022] DELETE /identity/v3/projects/099e35d20cd94455988579feb25288c9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:02.042550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539be9a-ef81-42e8-9a1b-25112ca8a7d1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.042978 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539be9a-ef81-42e8-9a1b-25112ca8a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.043199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539be9a-ef81-42e8-9a1b-25112ca8a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.067021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8539be9a-ef81-42e8-9a1b-25112ca8a7d1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:02.068263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8539be9a-ef81-42e8-9a1b-25112ca8a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:02.188174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 924/1876] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:02.195820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8fe22-7e43-4a3f-a47b-09e5ff96aaab None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.196113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8fe22-7e43-4a3f-a47b-09e5ff96aaab None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.196390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8fe22-7e43-4a3f-a47b-09e5ff96aaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.217471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8bb8fe22-7e43-4a3f-a47b-09e5ff96aaab None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:02.218759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8bb8fe22-7e43-4a3f-a47b-09e5ff96aaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:02.226810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 953/1877] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:02.231187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539be9a-ef81-42e8-9a1b-25112ca8a7d1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:02.245197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539be9a-ef81-42e8-9a1b-25112ca8a7d1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:02.252130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:02.252923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.253347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.253724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.255359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:02.256041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:02.258458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:02.291826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 925/1878] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:01:02.293212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8fe22-7e43-4a3f-a47b-09e5ff96aaab None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:02.303667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8fe22-7e43-4a3f-a47b-09e5ff96aaab None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:02.314261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:02.315064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.315608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.315958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.317369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:02.319167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:02.432327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 954/1879] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:02 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:02.435136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:02.445559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232267ad-0d8c-4627-8fb8-21cf79672272 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:02.478499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:02.516642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:02.517489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.517908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.518329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.522097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:02.522097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:02.622003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 926/1880] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:02 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:02.626307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:02.637783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7b10-3433-444e-a5a5-51d40c4e2d18 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:02.642734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:02.643306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.643630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.644031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.644762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:02.646424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:02.658333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 955/1881] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:02.662329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:02.674481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b7b64-f058-4876-9d35-08af00938488 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:02.679241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044fdcd-44d8-45b3-9938-475036eddb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:02.679951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044fdcd-44d8-45b3-9938-475036eddb62 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.680321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044fdcd-44d8-45b3-9938-475036eddb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.680602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044fdcd-44d8-45b3-9938-475036eddb62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.681305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044fdcd-44d8-45b3-9938-475036eddb62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:02.683521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044fdcd-44d8-45b3-9938-475036eddb62 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:02.693841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 927/1882] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:02.698640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:02.708784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42230b35-365c-4eed-b372-8010d7586fcd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:02.713734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c213c0-da8a-4e3a-a450-5281f279a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:02.714414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c213c0-da8a-4e3a-a450-5281f279a2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.714702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c213c0-da8a-4e3a-a450-5281f279a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.714983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c213c0-da8a-4e3a-a450-5281f279a2f7 admin admin] PATH_INFO: `/v3/projects/c7089377e1ee4c64973fe9d8a1b59735/users/19c483a587bf423aaa0fdb9b4e24ce9d/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.730627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c213c0-da8a-4e3a-a450-5281f279a2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7089377e1ee4c64973fe9d8a1b59735, user_id=19c483a587bf423aaa0fdb9b4e24ce9d, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:02.733900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c213c0-da8a-4e3a-a450-5281f279a2f7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:02.867554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 956/1883] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:02 2022] PUT /identity/v3/projects/c7089377e1ee4c64973fe9d8a1b59735/users/19c483a587bf423aaa0fdb9b4e24ce9d/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:02.873300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.873671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.874057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.894758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:02.896121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:02.896652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972826c4-b964-4e31-80fa-c4d6d70fe03f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.896946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972826c4-b964-4e31-80fa-c4d6d70fe03f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.897467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972826c4-b964-4e31-80fa-c4d6d70fe03f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.899108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 957/1884] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:01:02.926065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:02.926565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:02.926989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:02.953158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:02.955001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:03.009475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 928/1885] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:03.031437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:03.032893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:03.063248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 958/1886] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:01:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:03.081448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:03.082898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:03.100458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991e6b9-50ee-4f80-bd54-f1f89ff98f1d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:03.106244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c801f8-9ac1-4042-9104-05393a2aa42a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:03.107104 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c801f8-9ac1-4042-9104-05393a2aa42a service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:03.107390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c801f8-9ac1-4042-9104-05393a2aa42a service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:03.107701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c801f8-9ac1-4042-9104-05393a2aa42a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:03.117436 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d26402-1208-46e5-a3d1-9fe57de9b808 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:03.121410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c801f8-9ac1-4042-9104-05393a2aa42a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:03.126447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c801f8-9ac1-4042-9104-05393a2aa42a service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:03.126571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:03.126571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:03.126571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:03.126571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:03.137227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:03.139005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:03.145641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 959/1887] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:03.155714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 929/1888] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:04.772393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975ea0b-a18c-4414-9962-834d60b10dca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:04.772908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975ea0b-a18c-4414-9962-834d60b10dca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:04.772987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975ea0b-a18c-4414-9962-834d60b10dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:04.788431 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:04.814331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1975ea0b-a18c-4414-9962-834d60b10dca None None] MFA Rules not processed for user `19c483a587bf423aaa0fdb9b4e24ce9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:04.826185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1975ea0b-a18c-4414-9962-834d60b10dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:04.911055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 960/1889] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:04.923215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:04.925276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:04.988612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee28ea-8667-476c-a0a3-77f22ad0157c service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:04.992751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:04.993382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:04.993670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:04.994216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:05.005140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:05.006607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:05.010580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:05.016836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 930/1890] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:07.898093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a79bc-1d72-470c-872d-f1c69ad4bad8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:07.898771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a79bc-1d72-470c-872d-f1c69ad4bad8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:07.898771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a79bc-1d72-470c-872d-f1c69ad4bad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:07.919660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-739a79bc-1d72-470c-872d-f1c69ad4bad8 None None] MFA Rules not processed for user `19c483a587bf423aaa0fdb9b4e24ce9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:07.920308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-739a79bc-1d72-470c-872d-f1c69ad4bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:07.927669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 961/1891] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:15.381081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:17.913036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=32 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:20.038566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:20.040265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:20.065933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7737d-5d76-463a-86eb-d09189b318e8 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:20.067622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:20.068273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:20.068573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:20.068835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] PATH_INFO: `/v3/users/670637f19e32493181e52e9df6c2f576` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:20.070011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=670637f19e32493181e52e9df6c2f576)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:20.072800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:20.258253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670637f19e32493181e52e9df6c2f576'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:20.272029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670637f19e32493181e52e9df6c2f576'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:20.379296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670637f19e32493181e52e9df6c2f576'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:20.379965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670637f19e32493181e52e9df6c2f576'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:20.397775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670637f19e32493181e52e9df6c2f576'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:20.407038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 931/1892] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:20 2022] DELETE /identity/v3/users/670637f19e32493181e52e9df6c2f576 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:20.411936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:20.412412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:20.412521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:20.443024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:20.444614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:20.537904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 962/1893] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:20.548435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:20.550234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:20.609648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3d714-b945-4a9a-a943-fb4615eb1ed0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:20.614309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:20.614936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:20.615217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:20.615498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:20.629669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:20.631193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:20.641395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 932/1894] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:21.092635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:21.094414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:21.140387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe97bf-b61b-4c5f-89a2-852e63a17520 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:21.158606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:21.161063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:21.163053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:21.163668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] PATH_INFO: `/v3/projects/6a3a9bab8b8c4e48b5fc32e1452a591d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:21.167460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a3a9bab8b8c4e48b5fc32e1452a591d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:21.170268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:21.385654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a3a9bab8b8c4e48b5fc32e1452a591d', 'request_id': 'req-34484a24-eb83-4d5b-bbfb-c038777a904f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.458664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a3a9bab8b8c4e48b5fc32e1452a591d', 'request_id': 'req-34484a24-eb83-4d5b-bbfb-c038777a904f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.465229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a3a9bab8b8c4e48b5fc32e1452a591d', 'request_id': 'req-34484a24-eb83-4d5b-bbfb-c038777a904f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.476739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] The token cache is being invalidate because project 6a3a9bab8b8c4e48b5fc32e1452a591d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:21.477038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.477725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-34484a24-eb83-4d5b-bbfb-c038777a904f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.479466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 963/1895] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:21 2022] DELETE /identity/v3/projects/6a3a9bab8b8c4e48b5fc32e1452a591d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:21.483669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:21.494292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa91b02-9000-4353-af46-642b34a30820 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:21.562757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:21.563428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:21.563759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:21.564287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] PATH_INFO: `/v3/users/2ccbed04bb1b479dbb1aad7c8f59b165` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:21.565492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ccbed04bb1b479dbb1aad7c8f59b165)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:21.567504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:21.589386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:21.780313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ccbed04bb1b479dbb1aad7c8f59b165'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.788122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ccbed04bb1b479dbb1aad7c8f59b165'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.878716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ccbed04bb1b479dbb1aad7c8f59b165'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.880440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ccbed04bb1b479dbb1aad7c8f59b165'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.894272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bee54357-02a8-4f38-a328-04a15cb0eca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ccbed04bb1b479dbb1aad7c8f59b165'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:21.900799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 933/1896] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:21 2022] DELETE /identity/v3/users/2ccbed04bb1b479dbb1aad7c8f59b165 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:22.262416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4ab0-064d-4ff2-b7ea-d529e22e138d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:22.262416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4ab0-064d-4ff2-b7ea-d529e22e138d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:22.262416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4ab0-064d-4ff2-b7ea-d529e22e138d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:22.262416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 964/1897] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:01:22.281299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:22.282066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:22.282556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:22.310162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:22.311794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:22.415919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 934/1898] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:01:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:22.460765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec4ab0-064d-4ff2-b7ea-d529e22e138d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:22.466239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9eec4ab0-064d-4ff2-b7ea-d529e22e138d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:22.497105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec4ab0-064d-4ff2-b7ea-d529e22e138d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:22.506010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa42a28-2fce-4972-bb72-9dea9697a823 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:22.506717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42a28-2fce-4972-bb72-9dea9697a823 service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:22.507011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42a28-2fce-4972-bb72-9dea9697a823 service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:22.507294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42a28-2fce-4972-bb72-9dea9697a823 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:22.525820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa42a28-2fce-4972-bb72-9dea9697a823 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:22.529230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa42a28-2fce-4972-bb72-9dea9697a823 service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:22.539417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 965/1899] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:22.597710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:22.600339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:22.720858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943c994-ddab-4bb3-9dd9-abbc62022d00 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:22.726691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:22.727105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:22.727664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:22.728845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] PATH_INFO: `/v3/projects/ea946ed607ca43ffbb30f1622fc5b0b8` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:22.729149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea946ed607ca43ffbb30f1622fc5b0b8)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:22.731061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:22.877589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:22.934335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea946ed607ca43ffbb30f1622fc5b0b8', 'request_id': 'req-da41d82d-c32c-478c-b581-ab0ce8a25019'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:22.953712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea946ed607ca43ffbb30f1622fc5b0b8', 'request_id': 'req-da41d82d-c32c-478c-b581-ab0ce8a25019'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:22.970804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea946ed607ca43ffbb30f1622fc5b0b8', 'request_id': 'req-da41d82d-c32c-478c-b581-ab0ce8a25019'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:23.054847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] The token cache is being invalidate because project ea946ed607ca43ffbb30f1622fc5b0b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:23.055233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:23.058917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da41d82d-c32c-478c-b581-ab0ce8a25019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:23.058917 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 935/1900] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:22 2022] DELETE /identity/v3/projects/ea946ed607ca43ffbb30f1622fc5b0b8 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:23.063712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0d02-58a0-4ba3-acba-6267e9b01b24 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.067946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0d02-58a0-4ba3-acba-6267e9b01b24 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.068206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0d02-58a0-4ba3-acba-6267e9b01b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.091660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-acda0d02-58a0-4ba3-acba-6267e9b01b24 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:23.092907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-acda0d02-58a0-4ba3-acba-6267e9b01b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:23.215832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 966/1901] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:23.220911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8623d-40f3-4222-9d88-a9ec9fd68df8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.221293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8623d-40f3-4222-9d88-a9ec9fd68df8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.221604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8623d-40f3-4222-9d88-a9ec9fd68df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.245278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-34f8623d-40f3-4222-9d88-a9ec9fd68df8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:23.246560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34f8623d-40f3-4222-9d88-a9ec9fd68df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:23.256561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 936/1902] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:23.262009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda0d02-58a0-4ba3-acba-6267e9b01b24 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:23.273591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda0d02-58a0-4ba3-acba-6267e9b01b24 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:23.278861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:23.279695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.279695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.279928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.280493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:23.280805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:23.283166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:23.292557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:23.305136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 967/1903] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:01:23.306306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8623d-40f3-4222-9d88-a9ec9fd68df8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:23.316140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8623d-40f3-4222-9d88-a9ec9fd68df8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:23.325615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:23.325615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.325615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.325615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.325973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:23.325973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:23.450326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 937/1904] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:23 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:23.460342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:23.494460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a1e1-855a-464a-b958-b0bb374d1533 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:23.529265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:23.531029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:23.652942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:23.653583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.653865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.654139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.654810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:23.657017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:23.685400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d23a1-8c32-4f39-adf0-d2a20d21fa41 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:23.691086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:23.692094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.692547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.693273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] PATH_INFO: `/v3/users/a040a34d70e449e1b519e43fc6695668` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.694805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a040a34d70e449e1b519e43fc6695668)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:23.697391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:23.823520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 968/1905] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:23 2022] POST /identity/v3/users => generated 282 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:23.827651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:23.840423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b62c1-ebc4-4ef3-b72b-d7ff9a27a649 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:23.846004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:23.847400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.848053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.848533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.849457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:23.851809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:23.867144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 969/1906] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:23.872171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:23.885359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c132-0c6a-4926-b8a3-ab6841115de0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:23.890755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:23.891490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.892093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.892337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.893117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:23.894777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:23.906912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 970/1907] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:23.911735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:23.923189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0107378-cb09-488c-9f3c-1e8c39817a8f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:23.928377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f9811-d01e-4c6d-b69f-8ae364e34e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:23.934496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f9811-d01e-4c6d-b69f-8ae364e34e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:23.934978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f9811-d01e-4c6d-b69f-8ae364e34e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:23.935411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f9811-d01e-4c6d-b69f-8ae364e34e8c admin admin] PATH_INFO: `/v3/projects/5a1f7caa971940c68b532664182e2104/users/7e9580d237ad4e679e813c69a699ea73/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:23.952911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f9811-d01e-4c6d-b69f-8ae364e34e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1f7caa971940c68b532664182e2104, user_id=7e9580d237ad4e679e813c69a699ea73, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:23.955669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f9811-d01e-4c6d-b69f-8ae364e34e8c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:23.977257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a040a34d70e449e1b519e43fc6695668'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:23.987388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a040a34d70e449e1b519e43fc6695668'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.037365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 971/1908] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:23 2022] PUT /identity/v3/projects/5a1f7caa971940c68b532664182e2104/users/7e9580d237ad4e679e813c69a699ea73/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:24.039336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5c6f4-30b5-49ee-82f0-e485d35c4b31 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.040502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5c6f4-30b5-49ee-82f0-e485d35c4b31 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.040910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5c6f4-30b5-49ee-82f0-e485d35c4b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.086313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-98e5c6f4-30b5-49ee-82f0-e485d35c4b31 None None] MFA Rules not processed for user `7e9580d237ad4e679e813c69a699ea73`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:24.088424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-98e5c6f4-30b5-49ee-82f0-e485d35c4b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:24.117424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a040a34d70e449e1b519e43fc6695668'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.118173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a040a34d70e449e1b519e43fc6695668'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.136542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a040a34d70e449e1b519e43fc6695668'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.146739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 938/1909] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:23 2022] DELETE /identity/v3/users/a040a34d70e449e1b519e43fc6695668 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:24.177571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 972/1910] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:24.182211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:24.183673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:24.201107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:24.260169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d6a6-4b85-468b-b570-d31c93d78fe7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:24.266208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:24.267004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.268556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.268556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.268556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:24.270263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:24.382124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 939/1911] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:24 2022] POST /identity/v3/projects => generated 365 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:24.387233 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5c6f4-30b5-49ee-82f0-e485d35c4b31 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:24.445484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:24.491851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5c6f4-30b5-49ee-82f0-e485d35c4b31 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:24.510567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:24.521243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:24.584602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:24.590441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.590441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.590441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.590441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:24.590441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:24.618196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f128f8-451e-4480-9649-25d816409fde admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:24.625733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:24.626676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.627112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.627566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] PATH_INFO: `/v3/projects/ac9fc94b0d6e4c8d888716d39583dc39` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.630598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9fc94b0d6e4c8d888716d39583dc39)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:24.633153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:24.682087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 973/1912] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:24 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:24.686485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:24.700300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d468c1-8402-46dc-832f-d3e32e5116d7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:24.712957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:24.713950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.714513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.715081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.717402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:24.719321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:24.733558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 974/1913] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:24.742375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:24.757234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933cbd-9817-438e-a568-d10662dbb3fd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:24.763710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:24.768196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.768196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.768196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] PATH_INFO: `/v3/projects/3df53296968b40cd9a3c7c86319aa52d/users/ef8160e23bb7455b8183b4d52db5ceb0/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.787645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9fc94b0d6e4c8d888716d39583dc39', 'request_id': 'req-c07313a1-71a3-4acf-afb7-5328d69e11da'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.789361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df53296968b40cd9a3c7c86319aa52d, user_id=ef8160e23bb7455b8183b4d52db5ceb0, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:24.794842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:24.795566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9fc94b0d6e4c8d888716d39583dc39', 'request_id': 'req-c07313a1-71a3-4acf-afb7-5328d69e11da'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.802684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9fc94b0d6e4c8d888716d39583dc39', 'request_id': 'req-c07313a1-71a3-4acf-afb7-5328d69e11da'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.872668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] The token cache is being invalidate because project ac9fc94b0d6e4c8d888716d39583dc39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:24.873098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.873767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:24.873887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 975/1914] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:24 2022] PUT /identity/v3/projects/3df53296968b40cd9a3c7c86319aa52d/users/ef8160e23bb7455b8183b4d52db5ceb0/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:24.875848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 940/1915] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:24 2022] DELETE /identity/v3/projects/ac9fc94b0d6e4c8d888716d39583dc39 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:24.878691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:24.880201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:24.881958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:24.883049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:24.959032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07313a1-71a3-4acf-afb7-5328d69e11da admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:24.965022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:24.966439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.966912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.967433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] PATH_INFO: `/v3/users/0edfdfcb1a6a4b0baf8a70c9e3fb698a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.968184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e9a8-7f67-46a4-ada2-286f3958d651 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:24.969266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0edfdfcb1a6a4b0baf8a70c9e3fb698a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:24.972752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:24.976527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:24.978685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:24.978685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:24.978685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:24.979266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:24.981782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:25.000881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 976/1916] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:25.002014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:25.015846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69b7ea-97d2-4854-9f3f-e0fd322caa3e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:25.020689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49969a28-28f0-444d-8982-1f6682b9bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:25.021507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49969a28-28f0-444d-8982-1f6682b9bc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:25.021795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49969a28-28f0-444d-8982-1f6682b9bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:25.022081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49969a28-28f0-444d-8982-1f6682b9bc3b admin admin] PATH_INFO: `/v3/projects/3df53296968b40cd9a3c7c86319aa52d/users/ef8160e23bb7455b8183b4d52db5ceb0/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:25.024877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49969a28-28f0-444d-8982-1f6682b9bc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df53296968b40cd9a3c7c86319aa52d, user_id=ef8160e23bb7455b8183b4d52db5ceb0, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:25.027620 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49969a28-28f0-444d-8982-1f6682b9bc3b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:25.056821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 977/1917] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:25 2022] PUT /identity/v3/projects/3df53296968b40cd9a3c7c86319aa52d/users/ef8160e23bb7455b8183b4d52db5ceb0/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:25.058922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:25.059222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:25.060894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:25.074900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:25.100608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] MFA Rules not processed for user `ef8160e23bb7455b8183b4d52db5ceb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:25.102698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:25.148608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:25.156850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0edfdfcb1a6a4b0baf8a70c9e3fb698a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:25.165426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0edfdfcb1a6a4b0baf8a70c9e3fb698a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:25.209815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 978/1918] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:25.222114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:25.223699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:25.280495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0edfdfcb1a6a4b0baf8a70c9e3fb698a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:25.282554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0edfdfcb1a6a4b0baf8a70c9e3fb698a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:25.294283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96776bb1-5a25-49ee-a191-3d085b003be6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:25.303083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0edfdfcb1a6a4b0baf8a70c9e3fb698a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:25.316222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 941/1919] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:24 2022] DELETE /identity/v3/users/0edfdfcb1a6a4b0baf8a70c9e3fb698a => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:25.322262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:25.322262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:25.322801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:25.322957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:25.325843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:25.345219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:25.350782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:25.363496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 979/1920] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:25.629315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:25.631169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:25.710590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af91c0c-4ace-43a3-be19-46f61078c105 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:25.720236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff5f81-e362-4f9f-81de-97479b11014e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:25.720236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5f81-e362-4f9f-81de-97479b11014e service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:25.720236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5f81-e362-4f9f-81de-97479b11014e service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:25.720236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5f81-e362-4f9f-81de-97479b11014e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:25.720830 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c2ff5f81-e362-4f9f-81de-97479b11014e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:25.829442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff5f81-e362-4f9f-81de-97479b11014e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:25.831490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff5f81-e362-4f9f-81de-97479b11014e service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:25.842199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 942/1921] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:25 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:25.901839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:25.903355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:25.976671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd404e-969c-4744-a24d-cc2d9792ea87 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:26.009784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:26.034792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:26.046835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:26.047249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.047665 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.048073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] PATH_INFO: `/v3/projects/064a8d25e8164a4e9f5439a3d8613609` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:26.049415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] RBAC: Authorizing `identity:delete_project(project_id=064a8d25e8164a4e9f5439a3d8613609)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:26.051466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:26.213478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064a8d25e8164a4e9f5439a3d8613609', 'request_id': 'req-de0d29c3-e75f-47bd-b10e-8d9f025be05d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.325692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064a8d25e8164a4e9f5439a3d8613609', 'request_id': 'req-de0d29c3-e75f-47bd-b10e-8d9f025be05d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.332594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064a8d25e8164a4e9f5439a3d8613609', 'request_id': 'req-de0d29c3-e75f-47bd-b10e-8d9f025be05d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.364198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] The token cache is being invalidate because project 064a8d25e8164a4e9f5439a3d8613609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:26.364506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.366062 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.369474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 980/1922] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:25 2022] DELETE /identity/v3/projects/064a8d25e8164a4e9f5439a3d8613609 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:26.374505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd2ae-e2f0-472c-a641-bc74d52fa48f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.375018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd2ae-e2f0-472c-a641-bc74d52fa48f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.375452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd2ae-e2f0-472c-a641-bc74d52fa48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:26.421011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-177fd2ae-e2f0-472c-a641-bc74d52fa48f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:26.424527 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-177fd2ae-e2f0-472c-a641-bc74d52fa48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:26.543956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:26.543956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:26.553381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 943/1923] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:26.562363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7366a6-3281-4d3c-a9aa-c0628e68da00 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.562948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7366a6-3281-4d3c-a9aa-c0628e68da00 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.563364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7366a6-3281-4d3c-a9aa-c0628e68da00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:26.566955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d29c3-e75f-47bd-b10e-8d9f025be05d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:26.574023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:26.574713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.578432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.578739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] PATH_INFO: `/v3/users/06eb84f4363d4ffeaa2a6f317426b496` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:26.581844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06eb84f4363d4ffeaa2a6f317426b496)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:26.588428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:26.608199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-db7366a6-3281-4d3c-a9aa-c0628e68da00 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:26.609544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-db7366a6-3281-4d3c-a9aa-c0628e68da00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:26.641276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 944/1924] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:26.642952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7366a6-3281-4d3c-a9aa-c0628e68da00 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:26.653891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7366a6-3281-4d3c-a9aa-c0628e68da00 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:26.660453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:26.661454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.662042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.662628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:26.663707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:26.664268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:26.667445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:26.686652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 945/1925] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:01:26.693117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:26.704652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c3ca9-5994-47ee-80d7-5965d76210bd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:26.709982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:26.710721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.711105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.712033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:26.712937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:26.715041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:26.852700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 946/1926] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:26 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:26.853372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06eb84f4363d4ffeaa2a6f317426b496'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.856961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:26.935900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06eb84f4363d4ffeaa2a6f317426b496'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.943907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06eb84f4363d4ffeaa2a6f317426b496'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.945488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06eb84f4363d4ffeaa2a6f317426b496'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.946616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98fd8e77-5c9c-43bb-9080-59d8dc9302c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06eb84f4363d4ffeaa2a6f317426b496'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:26.967120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7050f7-3b99-458b-b4d3-154f57159d71 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:26.973540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:26.974496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.974879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.975254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:26.976112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:26.979267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:26.989372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 981/1927] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:26 2022] DELETE /identity/v3/users/06eb84f4363d4ffeaa2a6f317426b496 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:26.994637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896ab0-6470-4f61-b286-50551932024e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:26.995111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896ab0-6470-4f61-b286-50551932024e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:26.995391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896ab0-6470-4f61-b286-50551932024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:27.017594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-87896ab0-6470-4f61-b286-50551932024e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:27.018828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-87896ab0-6470-4f61-b286-50551932024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:27.057630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 947/1928] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:26 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:27.059037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:27.060954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:27.073492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:27.074757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b25f-71bc-4085-8701-e42de7ad5bba admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:27.102154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 982/1929] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:27.111828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87896ab0-6470-4f61-b286-50551932024e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:27.113312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-87896ab0-6470-4f61-b286-50551932024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:27.139785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:27.165272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:27.166131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:27.166539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:27.166945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:27.168274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:27.171427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:27.180476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87896ab0-6470-4f61-b286-50551932024e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:27.184622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 948/1930] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:27.185732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b39b6-3d99-4d6e-8b35-fb825790249a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:27.186810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b39b6-3d99-4d6e-8b35-fb825790249a service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:27.187316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b39b6-3d99-4d6e-8b35-fb825790249a service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:27.188985 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:27.200880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b39b6-3d99-4d6e-8b35-fb825790249a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:27.203821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e63919-184e-4ac6-8487-63ff9429b98d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:27.210626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:27.212545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:27.212835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:27.214304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:27.215046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:27.216987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:27.225437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b39b6-3d99-4d6e-8b35-fb825790249a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:27.228433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b39b6-3d99-4d6e-8b35-fb825790249a service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:27.236794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 949/1931] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:27.237776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:27.251739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca6205-282d-4b62-9825-68e733ca0062 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:27.258879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:27.260293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:27.263246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:27.263246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] PATH_INFO: `/v3/projects/27414fbc92604101b53665331df8a063/users/f6162744c6b24e1e850b2e7e34f288be/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:27.268892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 983/1932] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:27.291786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27414fbc92604101b53665331df8a063, user_id=f6162744c6b24e1e850b2e7e34f288be, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:27.295679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:27.408386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 950/1933] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:27 2022] PUT /identity/v3/projects/27414fbc92604101b53665331df8a063/users/f6162744c6b24e1e850b2e7e34f288be/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:27.421156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:27.421649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:27.421977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:27.518245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] MFA Rules not processed for user `f6162744c6b24e1e850b2e7e34f288be`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:27.520241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:27.638503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 984/1934] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:27.646313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:27.648485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:27.729194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbc998-2dd7-4e6e-84a6-4fa772c352f1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:27.734361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2ae6c-6af3-4c5f-843e-57bcd077e354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:27.735564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2ae6c-6af3-4c5f-843e-57bcd077e354 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:27.736008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2ae6c-6af3-4c5f-843e-57bcd077e354 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:27.736438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2ae6c-6af3-4c5f-843e-57bcd077e354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:27.749718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2ae6c-6af3-4c5f-843e-57bcd077e354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:27.752482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2ae6c-6af3-4c5f-843e-57bcd077e354 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:27.899909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 951/1935] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:27.956677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:27.956677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:28.042526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccbafb-593e-4ce8-848b-fc1bd6f9e0f5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:28.058335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:28.059860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.059860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.060072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] PATH_INFO: `/v3/projects/f1a23acae56c4f67a6da88bd3af7b61c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.062444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a23acae56c4f67a6da88bd3af7b61c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:28.064956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:28.155333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:28.171613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a23acae56c4f67a6da88bd3af7b61c', 'request_id': 'req-bd06c96d-5a64-4e99-9750-6f4f857b31bf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:28.236836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a23acae56c4f67a6da88bd3af7b61c', 'request_id': 'req-bd06c96d-5a64-4e99-9750-6f4f857b31bf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:28.242912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a23acae56c4f67a6da88bd3af7b61c', 'request_id': 'req-bd06c96d-5a64-4e99-9750-6f4f857b31bf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:28.254896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] The token cache is being invalidate because project f1a23acae56c4f67a6da88bd3af7b61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:28.255197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:28.255878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bd06c96d-5a64-4e99-9750-6f4f857b31bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:28.257419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 985/1936] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:27 2022] DELETE /identity/v3/projects/f1a23acae56c4f67a6da88bd3af7b61c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:28.262397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5a8f-482e-4ae4-965b-009e9f2e764a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.262694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5a8f-482e-4ae4-965b-009e9f2e764a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.262973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5a8f-482e-4ae4-965b-009e9f2e764a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.283929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0e9c5a8f-482e-4ae4-965b-009e9f2e764a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:28.285155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0e9c5a8f-482e-4ae4-965b-009e9f2e764a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:28.358234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 952/1937] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:28.359565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6098bd-3427-4a6d-a7a2-8b74a7417d89 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.360073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6098bd-3427-4a6d-a7a2-8b74a7417d89 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.360450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6098bd-3427-4a6d-a7a2-8b74a7417d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.385801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-aa6098bd-3427-4a6d-a7a2-8b74a7417d89 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:28.385801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aa6098bd-3427-4a6d-a7a2-8b74a7417d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:28.402714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 986/1938] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:28.405081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5a8f-482e-4ae4-965b-009e9f2e764a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:28.418998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5a8f-482e-4ae4-965b-009e9f2e764a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:28.424714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:28.425530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.425882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.426231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.427125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:28.427485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:28.429214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:28.445893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 953/1939] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:01:28.450049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6098bd-3427-4a6d-a7a2-8b74a7417d89 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:28.463087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6098bd-3427-4a6d-a7a2-8b74a7417d89 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:28.472390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:28.472390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.472390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.472390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.472719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:28.475016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:28.635883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 987/1940] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:28 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:28.639758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:28.659997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09768a9f-b757-4bf9-9272-8ae614c1a022 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:28.725143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:28.765556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:28.766148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.766438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.766716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.767397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:28.769448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:28.864550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 954/1941] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:28 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:28.868532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:28.878976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daad99d-4606-4c85-a1ad-3a84ed1103e6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:28.884162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:28.884776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.885050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.885354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.886675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:28.888362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:28.900526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 988/1942] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:28.904672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:28.915094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225bb5e-4dbb-41d4-bf4f-c4f39b5d72c5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:28.919733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44738ac-33bf-49f0-855e-7f49791ecdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:28.920565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44738ac-33bf-49f0-855e-7f49791ecdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.920847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44738ac-33bf-49f0-855e-7f49791ecdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.921120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44738ac-33bf-49f0-855e-7f49791ecdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.921833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44738ac-33bf-49f0-855e-7f49791ecdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:28.923451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44738ac-33bf-49f0-855e-7f49791ecdb5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:28.933982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 955/1943] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:28.938199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:28.948958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b107a5-db47-4266-8fd0-240e4bb6cbd7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:28.953618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:28.954362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:28.954649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:28.954928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] PATH_INFO: `/v3/projects/a08133eb7bcd4ffe81360e6ee2fcef21/users/16c12ba896024bc39c30fae66a51335e/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:28.969695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08133eb7bcd4ffe81360e6ee2fcef21, user_id=16c12ba896024bc39c30fae66a51335e, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:28.972400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:29.059957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 989/1944] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:28 2022] PUT /identity/v3/projects/a08133eb7bcd4ffe81360e6ee2fcef21/users/16c12ba896024bc39c30fae66a51335e/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:29.072274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:29.072565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:29.072843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:29.100354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] MFA Rules not processed for user `16c12ba896024bc39c30fae66a51335e`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:29.101578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:29.168196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 956/1945] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:29.179404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:29.181646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:29.241594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225f00-bf2c-4edf-9d60-c00cfff2885b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:29.245858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1baf4de-4ed0-4da6-aa18-0694181d93f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:29.246488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baf4de-4ed0-4da6-aa18-0694181d93f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:29.246771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baf4de-4ed0-4da6-aa18-0694181d93f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:29.247051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baf4de-4ed0-4da6-aa18-0694181d93f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:29.256395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1baf4de-4ed0-4da6-aa18-0694181d93f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:29.257846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1baf4de-4ed0-4da6-aa18-0694181d93f0 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:29.268666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 990/1946] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:39.164161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:39.264920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:42.858392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:42.859871 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:42.931246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088050b7-cf02-4799-a390-604bc92fb046 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:42.936324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:42.937863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:42.937863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:42.937863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] PATH_INFO: `/v3/users/f6162744c6b24e1e850b2e7e34f288be` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:42.938856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6162744c6b24e1e850b2e7e34f288be)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:42.941054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:43.130516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6162744c6b24e1e850b2e7e34f288be'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:43.138328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6162744c6b24e1e850b2e7e34f288be'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:43.234465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6162744c6b24e1e850b2e7e34f288be'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:43.235315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6162744c6b24e1e850b2e7e34f288be'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:43.252752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6162744c6b24e1e850b2e7e34f288be'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:43.264525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 957/1947] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:42 2022] DELETE /identity/v3/users/f6162744c6b24e1e850b2e7e34f288be => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:43.266029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:43.266515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:43.266609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:43.289645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:43.290896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:43.393107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 991/1948] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:43.423589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:43.425103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:43.514315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a82f58-5a5a-4ed0-9433-62964602fd78 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:43.518727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aaf84-0965-4d66-b716-6f9d426fdfa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:43.519580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aaf84-0965-4d66-b716-6f9d426fdfa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:43.520171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aaf84-0965-4d66-b716-6f9d426fdfa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:43.520661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aaf84-0965-4d66-b716-6f9d426fdfa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:43.530858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aaf84-0965-4d66-b716-6f9d426fdfa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:43.532717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aaf84-0965-4d66-b716-6f9d426fdfa3 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:43.544778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 958/1949] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:44.106884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:44.108394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:44.131803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f4cdc-68b3-46b7-ad2f-3bf5a5f466cd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:44.139168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:44.140503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.140893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.141619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] PATH_INFO: `/v3/projects/27414fbc92604101b53665331df8a063` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.143630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27414fbc92604101b53665331df8a063)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:44.145898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:44.280903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27414fbc92604101b53665331df8a063', 'request_id': 'req-8b1a2796-4169-46a3-924b-9dd780eeac07'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:44.371838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27414fbc92604101b53665331df8a063', 'request_id': 'req-8b1a2796-4169-46a3-924b-9dd780eeac07'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:44.379638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27414fbc92604101b53665331df8a063', 'request_id': 'req-8b1a2796-4169-46a3-924b-9dd780eeac07'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:44.394277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] The token cache is being invalidate because project 27414fbc92604101b53665331df8a063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:44.394712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:44.395401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8b1a2796-4169-46a3-924b-9dd780eeac07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:44.396982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 992/1950] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:44 2022] DELETE /identity/v3/projects/27414fbc92604101b53665331df8a063 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:44.402389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c6aa4-8c81-4a58-bdb8-1ad6bf3eb4fa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.402952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c6aa4-8c81-4a58-bdb8-1ad6bf3eb4fa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.403130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c6aa4-8c81-4a58-bdb8-1ad6bf3eb4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.424916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d87c6aa4-8c81-4a58-bdb8-1ad6bf3eb4fa None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:44.426305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d87c6aa4-8c81-4a58-bdb8-1ad6bf3eb4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:44.445362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:44.503741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 959/1951] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:44.507717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a11fb-f727-4366-b1ec-617bde8d6441 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.507717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a11fb-f727-4366-b1ec-617bde8d6441 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.507717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a11fb-f727-4366-b1ec-617bde8d6441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.527199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b57a11fb-f727-4366-b1ec-617bde8d6441 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:44.529090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b57a11fb-f727-4366-b1ec-617bde8d6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:44.539038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 993/1952] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:44.556584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c6aa4-8c81-4a58-bdb8-1ad6bf3eb4fa None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:44.566793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c6aa4-8c81-4a58-bdb8-1ad6bf3eb4fa None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:44.571732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:44.572370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.572654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.572930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.574873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:44.574873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:44.575587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:44.588476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 960/1953] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:01:44.592647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a11fb-f727-4366-b1ec-617bde8d6441 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:44.603739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a11fb-f727-4366-b1ec-617bde8d6441 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:44.608690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:44.609311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.609593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.609962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.610721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:44.612653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:44.725446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 994/1954] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:44 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:44.726610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:44.737749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9758dc-98aa-422d-a627-045802360ca3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:44.813529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:44.814165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.814484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.814763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.815507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:44.817613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:44.940068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 961/1955] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:44 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:44.944275 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:44.955330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bc61f-1954-403f-88f1-7e527f98ae82 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:44.960282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:44.960908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.961195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.961483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.962994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:44.964694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:44.980135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 995/1956] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:44.981307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:44.991644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ac363-f374-4cb2-91ac-f562bb794a1c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:44.996670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11040-8e77-41ea-8da3-2968911be53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:44.997303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11040-8e77-41ea-8da3-2968911be53f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:44.997619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11040-8e77-41ea-8da3-2968911be53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:44.998034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11040-8e77-41ea-8da3-2968911be53f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:44.998625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11040-8e77-41ea-8da3-2968911be53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:45.000298 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11040-8e77-41ea-8da3-2968911be53f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:45.012246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 962/1957] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:45.016408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:45.027212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6caf83-8d90-4931-915c-5d062469d949 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:45.031946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:45.032648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.032938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.033223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] PATH_INFO: `/v3/projects/9bc01f71de684af296c4e0af61550c52/users/ba6f6525c91148a5a3c6a5c0125c7157/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.049101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc01f71de684af296c4e0af61550c52, user_id=ba6f6525c91148a5a3c6a5c0125c7157, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:45.051792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:45.185898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 996/1958] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:45 2022] PUT /identity/v3/projects/9bc01f71de684af296c4e0af61550c52/users/ba6f6525c91148a5a3c6a5c0125c7157/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:45.196962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aa896-d9fd-4045-a0c9-dddbe8aa34b3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.197388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aa896-d9fd-4045-a0c9-dddbe8aa34b3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.197970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aa896-d9fd-4045-a0c9-dddbe8aa34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.233039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e73aa896-d9fd-4045-a0c9-dddbe8aa34b3 None None] MFA Rules not processed for user `ba6f6525c91148a5a3c6a5c0125c7157`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:45.234216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e73aa896-d9fd-4045-a0c9-dddbe8aa34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:45.305404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 963/1959] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:45.311456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:45.333714 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:45.375802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc17a32-e442-46f0-96f5-708dc6a49d64 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:45.381326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:45.381987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.382292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.382573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.383254 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:45.385857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:45.482512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 997/1960] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:45 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:45.485878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73aa896-d9fd-4045-a0c9-dddbe8aa34b3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:45.497329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73aa896-d9fd-4045-a0c9-dddbe8aa34b3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:45.510990 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:45.557295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:45.557928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.558215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.558495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.559174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:45.561160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:45.672420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 964/1961] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:45 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:45.675865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:45.686977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2eab-f31f-443f-a2dd-c4d8449a1f4f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:45.691522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:45.692181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.692472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.692858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.698096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:45.698096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:45.707330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 998/1962] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:45.711711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:45.723725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bcada-02dd-4bc4-9619-69ecb034703e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:45.730952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43181d95-a689-4209-90e5-491a3a371e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:45.733995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43181d95-a689-4209-90e5-491a3a371e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.733995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43181d95-a689-4209-90e5-491a3a371e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.733995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43181d95-a689-4209-90e5-491a3a371e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.733995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43181d95-a689-4209-90e5-491a3a371e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:45.734565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43181d95-a689-4209-90e5-491a3a371e52 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:45.749233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 965/1963] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:45.750672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:45.768341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7ac5-1a24-41f0-a7f2-33f9c628b0cd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:45.774290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:45.775266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.775584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.775879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] PATH_INFO: `/v3/projects/08f85cf36366414b98ff5dcd728796f2/users/e20f281324414e518257f2204718d2cb/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.793834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f85cf36366414b98ff5dcd728796f2, user_id=e20f281324414e518257f2204718d2cb, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:45.796936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:45.893458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 999/1964] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:45 2022] PUT /identity/v3/projects/08f85cf36366414b98ff5dcd728796f2/users/e20f281324414e518257f2204718d2cb/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:45.905018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2f56d-981f-4b3f-80d2-aaf39dfe7e43 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:45.905306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2f56d-981f-4b3f-80d2-aaf39dfe7e43 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:45.905631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2f56d-981f-4b3f-80d2-aaf39dfe7e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:45.934331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-74a2f56d-981f-4b3f-80d2-aaf39dfe7e43 None None] MFA Rules not processed for user `e20f281324414e518257f2204718d2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:45.935558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-74a2f56d-981f-4b3f-80d2-aaf39dfe7e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:45.998749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 966/1965] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:46.003277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.070631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc52b5-5fcc-4f17-bfad-44ce2ee12adb admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.075511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.076171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.076456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.076749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.077416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.079340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:46.146601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1000/1966] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:46 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:46.151328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2f56d-981f-4b3f-80d2-aaf39dfe7e43 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.163001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2f56d-981f-4b3f-80d2-aaf39dfe7e43 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.225515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.226346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.226738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.227112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.228670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.231121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:46.331890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 967/1967] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:46 2022] POST /identity/v3/users => generated 299 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:46.336284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.347464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2110a-fe6e-40c9-9c08-3bfa175d146c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.353046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.353749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.354037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.354313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.355019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.356671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:46.367936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1001/1968] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:46.383823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.394241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a484ad-cfa4-4a91-bf72-47539fc7458f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.398861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.400249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.400540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.400817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] PATH_INFO: `/v3/projects/918875fab314446d89227411b5426197/users/6116146a462e412cb6dc51dea4b25696/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.412864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:46.416388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918875fab314446d89227411b5426197, user_id=6116146a462e412cb6dc51dea4b25696, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.419309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:46.471117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 968/1969] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:46 2022] PUT /identity/v3/projects/918875fab314446d89227411b5426197/users/6116146a462e412cb6dc51dea4b25696/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:46.475106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.485886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46022edd-d18b-41e6-9b03-872d33ea4501 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.528202 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:46.542690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3835e8c-2d31-4e6a-89af-29c91afa41fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.545530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835e8c-2d31-4e6a-89af-29c91afa41fb admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.545530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835e8c-2d31-4e6a-89af-29c91afa41fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.545530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835e8c-2d31-4e6a-89af-29c91afa41fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.545530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3835e8c-2d31-4e6a-89af-29c91afa41fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.546499 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3835e8c-2d31-4e6a-89af-29c91afa41fb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:46.556311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1002/1970] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:46.560518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.571670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3ff3a-913a-47a4-8f7d-1fb239345ff2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.576541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.577211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.577549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.577874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] PATH_INFO: `/v3/projects/918875fab314446d89227411b5426197/users/6116146a462e412cb6dc51dea4b25696/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.579976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918875fab314446d89227411b5426197, user_id=6116146a462e412cb6dc51dea4b25696, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.582962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:46.594526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 969/1971] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:46 2022] PUT /identity/v3/projects/918875fab314446d89227411b5426197/users/6116146a462e412cb6dc51dea4b25696/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:46.605339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.605632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.606022 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.634426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] MFA Rules not processed for user `6116146a462e412cb6dc51dea4b25696`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:46.635782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:46.707286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1003/1972] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:46.719406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.721077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:46.780849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2578a1-8ed7-4431-bf66-6254bb5dd4e8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.784537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.785160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.785444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.785722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.819976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:46.822222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:46.854092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.856262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:46.867612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 970/1973] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:46.909881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dbf82-1fe5-4b98-8b3a-583b158ff56e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:46.916033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:46.916701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:46.917062 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:46.917341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] PATH_INFO: `/v3/users/16c12ba896024bc39c30fae66a51335e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:46.918689 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c12ba896024bc39c30fae66a51335e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:46.921122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:47.010818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:47.012647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:47.029807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551dce0-8d35-4043-8413-19523eee40be service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:47.034463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:47.035168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:47.035456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:47.035775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] PATH_INFO: `/v3/users/7e9580d237ad4e679e813c69a699ea73` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:47.036957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e9580d237ad4e679e813c69a699ea73)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:47.038875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:47.127284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c12ba896024bc39c30fae66a51335e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.282575 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c12ba896024bc39c30fae66a51335e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.291835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c12ba896024bc39c30fae66a51335e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.293344 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c12ba896024bc39c30fae66a51335e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.293344 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d9c9e8d6-17aa-4a17-aa81-f16c383203e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c12ba896024bc39c30fae66a51335e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.299069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:47.304847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e9580d237ad4e679e813c69a699ea73'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.317060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1004/1974] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:46 2022] DELETE /identity/v3/users/16c12ba896024bc39c30fae66a51335e => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:47.318446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e9580d237ad4e679e813c69a699ea73'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.318611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:47.318909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:47.319185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:47.338794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:47.340053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:47.404320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1005/1975] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:47.407710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e9580d237ad4e679e813c69a699ea73'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.408424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e9580d237ad4e679e813c69a699ea73'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.413917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:47.418185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:47.432108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c26984fb-32f2-41fc-8a33-91236aa7b460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e9580d237ad4e679e813c69a699ea73'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:47.444176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 971/1976] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:47 2022] DELETE /identity/v3/users/7e9580d237ad4e679e813c69a699ea73 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:47.449089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:47.450306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:47.450306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:47.458668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:47.471809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:47.473163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:47.495398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb718fa-33ff-42e6-8267-691bbeb7fced None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:47.540918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 972/1977] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:47.550417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:47.553496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:47.553496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:47.553496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:47.554866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:47.556620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:47.558350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:47.569337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:47.570655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa3257-2dc1-4588-a880-5395f067ea27 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:47.572054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:47.575688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:47.576384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:47.576761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:47.577127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:47.585514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1006/1978] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:47.588582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:47.590078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:47.600801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 973/1979] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:48.049263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:48.050885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:48.064666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2664-e21a-4b50-b103-c1a8a4dd59ed service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:48.070118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:48.070804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.071171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.071497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] PATH_INFO: `/v3/projects/a08133eb7bcd4ffe81360e6ee2fcef21` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.073220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08133eb7bcd4ffe81360e6ee2fcef21)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:48.074949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:48.100376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:48.102606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:48.116533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374e177-9b99-471b-91a4-ec57852a7e39 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:48.123429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:48.124312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.124762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.125042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] PATH_INFO: `/v3/projects/5a1f7caa971940c68b532664182e2104` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.126383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1f7caa971940c68b532664182e2104)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:48.128363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:48.203059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:48.210820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08133eb7bcd4ffe81360e6ee2fcef21', 'request_id': 'req-98d9161f-a5b2-4a06-8026-0c158c30a922'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.287187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08133eb7bcd4ffe81360e6ee2fcef21', 'request_id': 'req-98d9161f-a5b2-4a06-8026-0c158c30a922'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.294688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08133eb7bcd4ffe81360e6ee2fcef21', 'request_id': 'req-98d9161f-a5b2-4a06-8026-0c158c30a922'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.296366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1f7caa971940c68b532664182e2104', 'request_id': 'req-a6b79222-dbbf-4f71-b54a-03003db0461c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.303083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1f7caa971940c68b532664182e2104', 'request_id': 'req-a6b79222-dbbf-4f71-b54a-03003db0461c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.307146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] The token cache is being invalidate because project a08133eb7bcd4ffe81360e6ee2fcef21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:48.307454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.308171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-98d9161f-a5b2-4a06-8026-0c158c30a922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.309896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1007/1980] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:48 2022] DELETE /identity/v3/projects/a08133eb7bcd4ffe81360e6ee2fcef21 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:48.310664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1f7caa971940c68b532664182e2104', 'request_id': 'req-a6b79222-dbbf-4f71-b54a-03003db0461c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.315341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852733-828f-4839-9e66-935dda9b8460 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.315668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852733-828f-4839-9e66-935dda9b8460 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.315985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852733-828f-4839-9e66-935dda9b8460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.337194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-95852733-828f-4839-9e66-935dda9b8460 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:48.338546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-95852733-828f-4839-9e66-935dda9b8460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:48.403117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1008/1981] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:48.414348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2a98a-08ad-467c-b321-04ecc43be28f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.415584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2a98a-08ad-467c-b321-04ecc43be28f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.416903 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2a98a-08ad-467c-b321-04ecc43be28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.421235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] The token cache is being invalidate because project 5a1f7caa971940c68b532664182e2104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:48.421235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.421235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.425604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 974/1982] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:48 2022] DELETE /identity/v3/projects/5a1f7caa971940c68b532664182e2104 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:48.431006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:48.433978 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:48.442155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c2d2a98a-08ad-467c-b321-04ecc43be28f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:48.443393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c2d2a98a-08ad-467c-b321-04ecc43be28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:48.457050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1009/1983] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:48.459013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2a98a-08ad-467c-b321-04ecc43be28f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:48.465655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b79222-dbbf-4f71-b54a-03003db0461c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:48.470107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2a98a-08ad-467c-b321-04ecc43be28f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:48.473895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:48.474928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.475257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:48.475412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.475937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] PATH_INFO: `/v3/users/ef8160e23bb7455b8183b4d52db5ceb0` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.476191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.476572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.476969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.478494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:48.478619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef8160e23bb7455b8183b4d52db5ceb0)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:48.479200 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:48.481086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:48.482440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:48.497564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1010/1984] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:01:48.498969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:48.513388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304091-3ca9-4f60-b8a8-fda1ed7fc60b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:48.518746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:48.519658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.522201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.522201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.522201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:48.522920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:48.636703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1011/1985] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:48 2022] POST /identity/v3/projects => generated 361 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:48.638312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:48.706304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef8160e23bb7455b8183b4d52db5ceb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.719491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef8160e23bb7455b8183b4d52db5ceb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.750698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385dfed-e1e0-4bc2-a511-9a971369022d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:48.756402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:48.757277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:48.757764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:48.758717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:48.759624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:48.762097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:48.800342 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef8160e23bb7455b8183b4d52db5ceb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.801210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef8160e23bb7455b8183b4d52db5ceb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.809821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:48.824915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef8160e23bb7455b8183b4d52db5ceb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:48.835932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 975/1986] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:48 2022] DELETE /identity/v3/users/ef8160e23bb7455b8183b4d52db5ceb0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:48.906299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1012/1987] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:48 2022] POST /identity/v3/users => generated 281 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:48.910295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:48.911749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:48.995044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ee2a3-f4cf-4fbc-85fb-ff51658cae94 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.003764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:49.004793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.005151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.005514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.006322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:49.008406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:49.068578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 976/1988] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:49.068778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:49.068778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c94dd1-5765-4207-84ab-7b5caf6132f7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.068778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:49.068778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.069099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.069099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.069099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:49.069099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:49.071427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1013/1989] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:49.074940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:49.086578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368aee4-c08c-47b5-9ffb-c8aff57fc79b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.095947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15d155-7e86-4532-b0ff-436e46ea412e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:49.096660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15d155-7e86-4532-b0ff-436e46ea412e admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.096941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15d155-7e86-4532-b0ff-436e46ea412e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.097213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15d155-7e86-4532-b0ff-436e46ea412e admin admin] PATH_INFO: `/v3/projects/ab2bcdab80b64f6a8f799dbf4d0e99c5/users/5c14fc833c254ce9a77f3f444e715e10/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.118405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15d155-7e86-4532-b0ff-436e46ea412e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2bcdab80b64f6a8f799dbf4d0e99c5, user_id=5c14fc833c254ce9a77f3f444e715e10, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:49.121504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15d155-7e86-4532-b0ff-436e46ea412e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:49.146641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:49.148519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:49.209230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 977/1990] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:49 2022] PUT /identity/v3/projects/ab2bcdab80b64f6a8f799dbf4d0e99c5/users/5c14fc833c254ce9a77f3f444e715e10/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:49.229637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.229931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.230213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.264033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115881d9-b971-47e6-af03-e93bb78e6269 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.292127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:49.292632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] MFA Rules not processed for user `5c14fc833c254ce9a77f3f444e715e10`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:49.294139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:49.333135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:49.333907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.334263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.334550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.338502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:49.371513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 978/1991] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:49.406037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:49.407299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:49.413523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:49.415718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:49.427659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1014/1992] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:49.489928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057bd4c-2a94-4d5f-982a-ab6f8086ad68 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.494823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:49.495481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.495795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.496072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] PATH_INFO: `/v3/projects/3df53296968b40cd9a3c7c86319aa52d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.497248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df53296968b40cd9a3c7c86319aa52d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:49.499351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:49.650359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df53296968b40cd9a3c7c86319aa52d', 'request_id': 'req-9a49bcdc-e8f0-4075-85c9-14570c2032f6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:49.657001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df53296968b40cd9a3c7c86319aa52d', 'request_id': 'req-9a49bcdc-e8f0-4075-85c9-14570c2032f6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:49.663428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df53296968b40cd9a3c7c86319aa52d', 'request_id': 'req-9a49bcdc-e8f0-4075-85c9-14570c2032f6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:49.667646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:49.669643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:49.680857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8913758-8e77-4034-a352-827e51a8edb8 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.740992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:49.741719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.742011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.742293 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.785827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] The token cache is being invalidate because project 3df53296968b40cd9a3c7c86319aa52d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:49.785827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:49.785827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9a49bcdc-e8f0-4075-85c9-14570c2032f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:49.787271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 979/1993] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:49 2022] DELETE /identity/v3/projects/3df53296968b40cd9a3c7c86319aa52d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:49.792987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aad34-16f8-4141-8457-3fbdf675ea3c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.793180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aad34-16f8-4141-8457-3fbdf675ea3c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.793460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aad34-16f8-4141-8457-3fbdf675ea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.834996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a48aad34-16f8-4141-8457-3fbdf675ea3c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:49.837908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a48aad34-16f8-4141-8457-3fbdf675ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:49.850920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:49.853013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:49.864631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1015/1994] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:49.885307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:49.927459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 980/1995] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:49.935738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf16b9-fa2e-4295-8e2e-9c4b59c67a4e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.936031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf16b9-fa2e-4295-8e2e-9c4b59c67a4e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.936303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf16b9-fa2e-4295-8e2e-9c4b59c67a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:49.974192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d2bf16b9-fa2e-4295-8e2e-9c4b59c67a4e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:49.975594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d2bf16b9-fa2e-4295-8e2e-9c4b59c67a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:49.984519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 981/1996] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:49.988533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf16b9-fa2e-4295-8e2e-9c4b59c67a4e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:49.990868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c085e7-3324-40db-8b5c-9e85b255a128 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.997523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:49.998154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf16b9-fa2e-4295-8e2e-9c4b59c67a4e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:49.998385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:49.998784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:49.999177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] PATH_INFO: `/v3/users/5c14fc833c254ce9a77f3f444e715e10` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.000842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c14fc833c254ce9a77f3f444e715e10)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.003209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.007104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.008050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.021190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.021190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.021190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:50.021190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.021190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.027565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 982/1997] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:01:50.031380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.042145 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1bcc2-f1f5-4dbc-921c-e4bd50a43492 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.047049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.047699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.047981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.048255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.048943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.050834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.155005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 983/1998] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:50 2022] POST /identity/v3/projects => generated 399 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:50.157252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.216086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c14fc833c254ce9a77f3f444e715e10'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:50.220631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01d4b9-bf92-4d7a-9133-9d031cfeed8d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.225666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.226293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.226581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.227491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.228379 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.230393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.235632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:50.316341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c14fc833c254ce9a77f3f444e715e10'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:50.322382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 984/1999] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:50 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:50.324585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c14fc833c254ce9a77f3f444e715e10'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:50.325289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c14fc833c254ce9a77f3f444e715e10'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:50.325882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d6a9a2b5-d50d-4e1c-b249-988072e3e823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c14fc833c254ce9a77f3f444e715e10'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:50.326427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.328127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:50.340942 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483a8d9-1e04-4f92-9534-e221b6bad7b0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.344057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1016/2000] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:49 2022] DELETE /identity/v3/users/5c14fc833c254ce9a77f3f444e715e10 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:50.345918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.346535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.346923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.347209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.347949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.348303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.348627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.348961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.349709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.360096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 985/2001] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:50.364258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.369636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:50.370864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:50.374053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecacbaf-d48d-4a46-a7bf-02bfad98bc81 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.432767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1017/2002] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:50.435188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.435854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.436148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.436540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.437109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.439371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.442565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.443960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:50.453407 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 986/2003] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:50.454444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.464942 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01164-c582-4cb8-ae96-b1d490bd9075 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.470387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.471140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.471431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.471743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] PATH_INFO: `/v3/projects/bf8b7379cfcc49d19be805864bf42919/users/13dbc33c06d54b80a3f2f380fcc2b134/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.487323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf8b7379cfcc49d19be805864bf42919, user_id=13dbc33c06d54b80a3f2f380fcc2b134, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.490134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.505471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a7007-0316-46f1-9a11-0db9058d0d05 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.510111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ae5aa-8e72-4fbd-b418-3592ebb0a91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.511000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ae5aa-8e72-4fbd-b418-3592ebb0a91a service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.511000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ae5aa-8e72-4fbd-b418-3592ebb0a91a service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.511283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ae5aa-8e72-4fbd-b418-3592ebb0a91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.520597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ae5aa-8e72-4fbd-b418-3592ebb0a91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.522049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ae5aa-8e72-4fbd-b418-3592ebb0a91a service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.532488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1018/2004] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:50.565916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 987/2005] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:50 2022] PUT /identity/v3/projects/bf8b7379cfcc49d19be805864bf42919/users/13dbc33c06d54b80a3f2f380fcc2b134/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:50.578099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91158a1d-c96d-4718-aa6c-bb8a154006e2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.578395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91158a1d-c96d-4718-aa6c-bb8a154006e2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.578676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91158a1d-c96d-4718-aa6c-bb8a154006e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.584244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:50.610365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-91158a1d-c96d-4718-aa6c-bb8a154006e2 None None] MFA Rules not processed for user `13dbc33c06d54b80a3f2f380fcc2b134`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:50.611770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-91158a1d-c96d-4718-aa6c-bb8a154006e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:50.677932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1019/2006] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:50.683063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.752274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b48f-47d0-404a-93f7-9d72c68860f1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.757192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.758966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.759274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.759456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.760151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.762251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:50.836537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 988/2007] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:50 2022] POST /identity/v3/projects => generated 401 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:50.840676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91158a1d-c96d-4718-aa6c-bb8a154006e2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.851931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91158a1d-c96d-4718-aa6c-bb8a154006e2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:50.897539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:50.899337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:50.902422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:50.953547 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:50.954516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:50.954836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:50.955094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:50.955994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:50.958883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.004927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9976a29-668a-4afd-ae19-00ca4f4a2a6d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.010507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.011084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.011421 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.011731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] PATH_INFO: `/v3/projects/ab2bcdab80b64f6a8f799dbf4d0e99c5` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.012906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2bcdab80b64f6a8f799dbf4d0e99c5)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.014841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.090638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1020/2008] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:50 2022] POST /identity/v3/users => generated 307 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:51.091554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.095128 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:51.125970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2bcdab80b64f6a8f799dbf4d0e99c5', 'request_id': 'req-bb467558-71cc-4fbf-8606-3cc3368faaf8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:51.134498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2bcdab80b64f6a8f799dbf4d0e99c5', 'request_id': 'req-bb467558-71cc-4fbf-8606-3cc3368faaf8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:51.137706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:51.146666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2bcdab80b64f6a8f799dbf4d0e99c5', 'request_id': 'req-bb467558-71cc-4fbf-8606-3cc3368faaf8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:51.179363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792602b-4928-4dbb-9e5d-6f90284ea45a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.238943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a27c1-4a98-43ce-86cb-b04a17733464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.240082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a27c1-4a98-43ce-86cb-b04a17733464 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.240695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a27c1-4a98-43ce-86cb-b04a17733464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.241532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a27c1-4a98-43ce-86cb-b04a17733464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.242267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] The token cache is being invalidate because project ab2bcdab80b64f6a8f799dbf4d0e99c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:51.242884 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:51.244041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:51.244894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-140a27c1-4a98-43ce-86cb-b04a17733464 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:51.246085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 989/2009] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:50 2022] DELETE /identity/v3/projects/ab2bcdab80b64f6a8f799dbf4d0e99c5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:51.249349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.250832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:51.322406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb467558-71cc-4fbf-8606-3cc3368faaf8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.324019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a27c1-4a98-43ce-86cb-b04a17733464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.325654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a27c1-4a98-43ce-86cb-b04a17733464 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.327666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.328307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.328597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.328881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.329588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.331199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.339972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1021/2010] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:51.341264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ac53b-12db-4b43-858a-653daf6d04eb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.341554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ac53b-12db-4b43-858a-653daf6d04eb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.341834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ac53b-12db-4b43-858a-653daf6d04eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.343600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 990/2011] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:51.347169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.359372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a750dd-c013-40ba-b8b5-126d824af818 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.368200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.368893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.370631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.370631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] PATH_INFO: `/v3/projects/b244817dce4241f7b1851f010f00030c/users/6f53db7506ce441d923fcaac902a34ab/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.376535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e63ac53b-12db-4b43-858a-653daf6d04eb None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:51.377766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e63ac53b-12db-4b43-858a-653daf6d04eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:51.387290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1022/2012] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:51.393213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b244817dce4241f7b1851f010f00030c, user_id=6f53db7506ce441d923fcaac902a34ab, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.396158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.401129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277206-d1e1-49ff-b80e-de21bb2133d6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.401426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277206-d1e1-49ff-b80e-de21bb2133d6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.401702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277206-d1e1-49ff-b80e-de21bb2133d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.421472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-31277206-d1e1-49ff-b80e-de21bb2133d6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:51.422730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-31277206-d1e1-49ff-b80e-de21bb2133d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:51.431269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1023/2013] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:51.435363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277206-d1e1-49ff-b80e-de21bb2133d6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.444274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277206-d1e1-49ff-b80e-de21bb2133d6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.449510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.450120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.450409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.450685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.451387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:01:51.451764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.453374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.465413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1024/2014] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:01:51.469417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.479295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17a826-dfd1-44ed-a881-d49328e03c48 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.483998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.484626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.484914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.485195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.485864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.487806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.505314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 991/2015] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:51 2022] PUT /identity/v3/projects/b244817dce4241f7b1851f010f00030c/users/6f53db7506ce441d923fcaac902a34ab/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:51.508961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.569750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76691469-4bfb-4cc4-885b-5d95aedb543d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.575246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.575884 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.576167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.576439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.577241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.578853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.589025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 992/2016] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:01:51.593169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.604106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0722b-7448-4b53-8abd-ff2e414ea312 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.609197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a78e7-da7a-4af3-87eb-d8aa7c8ed724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.613139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a78e7-da7a-4af3-87eb-d8aa7c8ed724 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.613260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1025/2017] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:51 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:51.613367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a78e7-da7a-4af3-87eb-d8aa7c8ed724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.613367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a78e7-da7a-4af3-87eb-d8aa7c8ed724 admin admin] PATH_INFO: `/v3/projects/b244817dce4241f7b1851f010f00030c/users/6f53db7506ce441d923fcaac902a34ab/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.613367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a78e7-da7a-4af3-87eb-d8aa7c8ed724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b244817dce4241f7b1851f010f00030c, user_id=6f53db7506ce441d923fcaac902a34ab, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.614283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.615079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a78e7-da7a-4af3-87eb-d8aa7c8ed724 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.629121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 993/2018] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:51 2022] PUT /identity/v3/projects/b244817dce4241f7b1851f010f00030c/users/6f53db7506ce441d923fcaac902a34ab/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:51.641278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.641571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.641837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.675837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] MFA Rules not processed for user `6f53db7506ce441d923fcaac902a34ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:51.677189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:51.684987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4a8cf-1c00-4b5e-b218-32ca200c0ff1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.740493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.741121 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.741404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.741682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.742972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.745046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.748201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 994/2019] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:51.750206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:51.760927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.762306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:51.819719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2186a27-4e79-43af-8819-eaf9645d008f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.823455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.824122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.824435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.824727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.834068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.835551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.845176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 995/2020] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:51.856102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1026/2021] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:51 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:01:51.857385 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.868412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454d1dd-2439-47aa-aea3-c24b3574b7c6 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.873193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.874877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.874877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.874877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.875147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.876775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.887003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 996/2022] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:51.891053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.901365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edcf34-ed9a-4b49-8b2b-69b771162aad admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.906094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.906701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.906989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.907266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.908102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.909739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:51.920694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1027/2023] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:51.924688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:51.935252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d550fe1-bb5e-4abf-a540-255b860ffb06 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:51.940073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e1d76-983f-4a11-b96e-429c55778b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:51.940771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e1d76-983f-4a11-b96e-429c55778b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:51.941061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e1d76-983f-4a11-b96e-429c55778b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:51.941339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e1d76-983f-4a11-b96e-429c55778b7b admin admin] PATH_INFO: `/v3/projects/d488606d5f2141bc813b2ec306e571d7/users/bd58c88d777d4e579278f8c87e3c4042/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:51.969697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e1d76-983f-4a11-b96e-429c55778b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d488606d5f2141bc813b2ec306e571d7, user_id=bd58c88d777d4e579278f8c87e3c4042, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:51.973617 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e1d76-983f-4a11-b96e-429c55778b7b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:52.098165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:52.116989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 997/2024] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:51 2022] PUT /identity/v3/projects/d488606d5f2141bc813b2ec306e571d7/users/bd58c88d777d4e579278f8c87e3c4042/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:52.120199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:52.127252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d86f-15bf-4147-9d7a-2e0650a5b1a7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:52.129637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d86f-15bf-4147-9d7a-2e0650a5b1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:52.129637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d86f-15bf-4147-9d7a-2e0650a5b1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:52.136069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705064e0-98b1-4e21-b176-f435bab359c5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:52.168943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-46d9d86f-15bf-4147-9d7a-2e0650a5b1a7 None None] MFA Rules not processed for user `bd58c88d777d4e579278f8c87e3c4042`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:52.170676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-46d9d86f-15bf-4147-9d7a-2e0650a5b1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:52.216805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:52.217363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:52.217649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:52.217928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:52.220092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:52.264176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 998/2025] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:52.267773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9d86f-15bf-4147-9d7a-2e0650a5b1a7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:52.286567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:52.327887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:52.327887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:52.337445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1028/2026] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:52.351971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9d86f-15bf-4147-9d7a-2e0650a5b1a7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:52.357204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:52.357973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:52.358324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:52.358663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:52.359411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:52.361650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:52.498078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 999/2027] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:01:52 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:52.499639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:52.516320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134bfd-ca7a-4a66-9e7b-5ce841988ecf service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:52.585320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:52.587113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:52.587418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:52.587746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:52.588862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:52.592597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:52.699395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:52.705197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1029/2028] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:01:52 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:01:52.710512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:52.722285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c7fa3-b488-4f21-9565-e2f11d649937 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:52.727485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:52.728042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:52.728330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:52.728606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:52.729394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:52.731031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:52.741809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1000/2029] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:52.746054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:52.756462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c71f6-03de-46db-a57f-e1ce78789d14 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:52.762821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:52.763777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:52.764016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:52.764305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] PATH_INFO: `/v3/projects/c912ce0dc0b842af95fe71ddea4e6e06/users/6383385603244e4699ae206911b1157c/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:52.782410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c912ce0dc0b842af95fe71ddea4e6e06, user_id=6383385603244e4699ae206911b1157c, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:52.785401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:52.884063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1030/2030] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:52 2022] PUT /identity/v3/projects/c912ce0dc0b842af95fe71ddea4e6e06/users/6383385603244e4699ae206911b1157c/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:52.889118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:52.924568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49c19c-ab03-4731-ac6d-b410c5452c3f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:53.025698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6479b2-53f8-4290-81f3-c6ba09865869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:53.026340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6479b2-53f8-4290-81f3-c6ba09865869 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:53.026761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6479b2-53f8-4290-81f3-c6ba09865869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:53.027039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6479b2-53f8-4290-81f3-c6ba09865869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:53.027796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6479b2-53f8-4290-81f3-c6ba09865869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:53.029505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6479b2-53f8-4290-81f3-c6ba09865869 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:53.041589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1001/2031] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:01:53.050621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:53.058746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd30025-5d5a-4cf6-bce1-5f373e29bb88 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:53.063725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:53.064422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:53.064692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:53.064971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] PATH_INFO: `/v3/projects/c912ce0dc0b842af95fe71ddea4e6e06/users/6383385603244e4699ae206911b1157c/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:53.067000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c912ce0dc0b842af95fe71ddea4e6e06, user_id=6383385603244e4699ae206911b1157c, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:53.087984 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:53.098951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1031/2032] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:01:53 2022] PUT /identity/v3/projects/c912ce0dc0b842af95fe71ddea4e6e06/users/6383385603244e4699ae206911b1157c/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:53.116684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c157ba8-68e0-457a-bc2f-8654cedadfc5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:53.116684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c157ba8-68e0-457a-bc2f-8654cedadfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:53.116684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c157ba8-68e0-457a-bc2f-8654cedadfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:53.126185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:53.158199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9c157ba8-68e0-457a-bc2f-8654cedadfc5 None None] MFA Rules not processed for user `6383385603244e4699ae206911b1157c`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:53.159495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9c157ba8-68e0-457a-bc2f-8654cedadfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:53.244350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1002/2033] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:53.289606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:53.357645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28643e66-ded7-4f8e-980b-ae8056b95dd7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:53.361387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:53.362054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:53.362341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:53.362619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:53.419089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:53.420553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:53.429411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1032/2034] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:54.344680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:54.693143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:54.780271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d21fd-9cc4-4c00-8fb0-290151b56681 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:54.780558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d21fd-9cc4-4c00-8fb0-290151b56681 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:54.780837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d21fd-9cc4-4c00-8fb0-290151b56681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:54.801087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-495d21fd-9cc4-4c00-8fb0-290151b56681 None None] MFA Rules not processed for user `19c483a587bf423aaa0fdb9b4e24ce9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:54.802325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-495d21fd-9cc4-4c00-8fb0-290151b56681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:54.861807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1003/2035] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:57.583153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:57.584737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:57.603918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:57.656628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f4d9-93c5-4887-8092-63ba6edd7c5e service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:57.665188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:57.665188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:57.665188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:57.665188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] PATH_INFO: `/v3/users/ba6f6525c91148a5a3c6a5c0125c7157` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:57.665638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6f6525c91148a5a3c6a5c0125c7157)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:57.667208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:57.765588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d21fd-9cc4-4c00-8fb0-290151b56681 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:57.779138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d21fd-9cc4-4c00-8fb0-290151b56681 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:57.784129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:57.784843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:57.785278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:57.785556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] PATH_INFO: `/v3/users/bd58c88d777d4e579278f8c87e3c4042` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:57.791564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd58c88d777d4e579278f8c87e3c4042)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:57.795349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:57.879701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6f6525c91148a5a3c6a5c0125c7157'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:57.955920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6f6525c91148a5a3c6a5c0125c7157'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:57.964816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6f6525c91148a5a3c6a5c0125c7157'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:57.965663 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6f6525c91148a5a3c6a5c0125c7157'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:57.966287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4cc3e813-bae2-4720-8b4b-7cb4bfa3c42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6f6525c91148a5a3c6a5c0125c7157'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:57.990622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1033/2036] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:57 2022] DELETE /identity/v3/users/ba6f6525c91148a5a3c6a5c0125c7157 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:57.993040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078743-e839-47e4-b069-73d297d8f603 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:57.993420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078743-e839-47e4-b069-73d297d8f603 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:57.993760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078743-e839-47e4-b069-73d297d8f603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:58.003323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd58c88d777d4e579278f8c87e3c4042'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:58.013217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd58c88d777d4e579278f8c87e3c4042'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:58.015389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9a078743-e839-47e4-b069-73d297d8f603 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:58.016780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9a078743-e839-47e4-b069-73d297d8f603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:58.104392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd58c88d777d4e579278f8c87e3c4042'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:58.105316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd58c88d777d4e579278f8c87e3c4042'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:58.127893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:58.131766 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd58c88d777d4e579278f8c87e3c4042'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:58.142240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1034/2038] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:58.143267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1004/2038] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:57 2022] DELETE /identity/v3/users/bd58c88d777d4e579278f8c87e3c4042 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:01:58.144994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:58.145526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:58.146070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:58.154233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:58.156261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:58.177274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:58.178864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:58.224554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:58.251169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f3218-15db-4488-8101-1065ec4e1901 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:58.266550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499290ef-d4a0-417f-8f95-e35bf8c19931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:58.266550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499290ef-d4a0-417f-8f95-e35bf8c19931 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:58.266789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499290ef-d4a0-417f-8f95-e35bf8c19931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:58.267074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499290ef-d4a0-417f-8f95-e35bf8c19931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:58.307251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1035/2039] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:01:58.320946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:58.323196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:58.340561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d9ff6-db4b-434e-8b90-bcf09d21210e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:58.345957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:58.346883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:58.347334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:58.347795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:58.363456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:58.366282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:58.381255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1036/2040] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:58.387624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499290ef-d4a0-417f-8f95-e35bf8c19931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:58.398643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499290ef-d4a0-417f-8f95-e35bf8c19931 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:58.402569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1005/2041] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:01:58.665933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:58.668447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:58.687466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315c5ba-858b-40f1-ad49-1df4293a2bd3 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:58.693925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:58.699386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:58.699386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:58.699386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] PATH_INFO: `/v3/users/13dbc33c06d54b80a3f2f380fcc2b134` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:58.700435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13dbc33c06d54b80a3f2f380fcc2b134)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:58.704133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:58.951434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13dbc33c06d54b80a3f2f380fcc2b134'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.011635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13dbc33c06d54b80a3f2f380fcc2b134'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.024117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13dbc33c06d54b80a3f2f380fcc2b134'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.024873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13dbc33c06d54b80a3f2f380fcc2b134'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.025580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13dbc33c06d54b80a3f2f380fcc2b134'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.066612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1037/2042] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:58 2022] DELETE /identity/v3/users/13dbc33c06d54b80a3f2f380fcc2b134 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:59.070812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:59.071348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:59.071348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:59.104031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:01:59.105346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:59.216655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1006/2043] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:01:59.239806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:59.241292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:59.271800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:59.275232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:59.307302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030525e-682c-47e2-a299-20a1abd16c7f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:59.323470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:59.324255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:59.324681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:59.324966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] PATH_INFO: `/v3/projects/d488606d5f2141bc813b2ec306e571d7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:59.328354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d488606d5f2141bc813b2ec306e571d7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:59.330478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:59.337366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36120-ed65-4d51-a051-3c57eb41f4e4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:59.341931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:59.342625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:59.342943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:59.343228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:59.366291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:59.371051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:59.383254 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1038/2044] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:01:59.384157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:59.385603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:59.402349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6d64d-cf78-4b8f-91a4-835f34a55ffd service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:59.407924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:59.408866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:59.409157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:59.409435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] PATH_INFO: `/v3/projects/9bc01f71de684af296c4e0af61550c52` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:59.410661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc01f71de684af296c4e0af61550c52)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:59.412638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:59.493648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:59.501462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd488606d5f2141bc813b2ec306e571d7', 'request_id': 'req-6421ceef-0da3-4ea5-af2c-b76c010996bb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.509670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd488606d5f2141bc813b2ec306e571d7', 'request_id': 'req-6421ceef-0da3-4ea5-af2c-b76c010996bb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.515803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd488606d5f2141bc813b2ec306e571d7', 'request_id': 'req-6421ceef-0da3-4ea5-af2c-b76c010996bb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.596502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:01:59.609004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc01f71de684af296c4e0af61550c52', 'request_id': 'req-5cf4e530-9d73-4634-ae69-bb90a681bd4e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.637991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] The token cache is being invalidate because project d488606d5f2141bc813b2ec306e571d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:59.640283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.640283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.642118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1007/2045] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:59 2022] DELETE /identity/v3/projects/d488606d5f2141bc813b2ec306e571d7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:59.646444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:59.648008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:59.721816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421ceef-0da3-4ea5-af2c-b76c010996bb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:59.767280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc01f71de684af296c4e0af61550c52', 'request_id': 'req-5cf4e530-9d73-4634-ae69-bb90a681bd4e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.774759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc01f71de684af296c4e0af61550c52', 'request_id': 'req-5cf4e530-9d73-4634-ae69-bb90a681bd4e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.786991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] The token cache is being invalidate because project 9bc01f71de684af296c4e0af61550c52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:01:59.787291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.788050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:01:59.793442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1039/2046] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:01:59 2022] DELETE /identity/v3/projects/9bc01f71de684af296c4e0af61550c52 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:01:59.794523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:01:59.795941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:01:59.799636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:59.800492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:59.801220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:59.801889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] PATH_INFO: `/v3/users/6383385603244e4699ae206911b1157c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:59.810634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6383385603244e4699ae206911b1157c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:59.813428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:01:59.815420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e530-9d73-4634-ae69-bb90a681bd4e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:01:59.822673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:01:59.823329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:01:59.823690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:01:59.823968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] PATH_INFO: `/v3/users/e20f281324414e518257f2204718d2cb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:01:59.825165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20f281324414e518257f2204718d2cb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:01:59.827195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:00.029128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6383385603244e4699ae206911b1157c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.053060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6383385603244e4699ae206911b1157c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.065824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20f281324414e518257f2204718d2cb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.133067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6383385603244e4699ae206911b1157c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.134359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6383385603244e4699ae206911b1157c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.148891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:00.150112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6383385603244e4699ae206911b1157c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.156886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1008/2047] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:59 2022] DELETE /identity/v3/users/6383385603244e4699ae206911b1157c => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:00.206831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20f281324414e518257f2204718d2cb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.218286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20f281324414e518257f2204718d2cb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.218941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20f281324414e518257f2204718d2cb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.219839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20f281324414e518257f2204718d2cb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.244911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1040/2048] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:01:59 2022] DELETE /identity/v3/users/e20f281324414e518257f2204718d2cb => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:00.411770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:00.413309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:00.548376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac3603-e422-4715-aeb8-1d28b6ec1ef5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:00.667441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:00.669096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:00.669718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:00.669994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] PATH_INFO: `/v3/projects/bf8b7379cfcc49d19be805864bf42919` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:00.671786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf8b7379cfcc49d19be805864bf42919)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:00.673868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:00.868671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf8b7379cfcc49d19be805864bf42919', 'request_id': 'req-db422953-9b75-402c-95b9-5a3947a71fd8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.875687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf8b7379cfcc49d19be805864bf42919', 'request_id': 'req-db422953-9b75-402c-95b9-5a3947a71fd8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:00.883778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf8b7379cfcc49d19be805864bf42919', 'request_id': 'req-db422953-9b75-402c-95b9-5a3947a71fd8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.007021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] The token cache is being invalidate because project bf8b7379cfcc49d19be805864bf42919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:01.007021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.007021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.007021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1009/2049] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:00 2022] DELETE /identity/v3/projects/bf8b7379cfcc49d19be805864bf42919 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:01.011180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:01.012723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:01.037566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2a759-ff85-4c8e-9e0f-fa940d40342a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:01.103812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:01.104396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:01.104688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:01.104965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] PATH_INFO: `/v3/users/6f53db7506ce441d923fcaac902a34ab` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:01.106463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f53db7506ce441d923fcaac902a34ab)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:01.108604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:01.179353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:01.180933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:01.199321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db422953-9b75-402c-95b9-5a3947a71fd8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:01.205069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:01.206186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:01.206777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:01.207744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] PATH_INFO: `/v3/projects/08f85cf36366414b98ff5dcd728796f2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:01.209486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f85cf36366414b98ff5dcd728796f2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:01.213282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:01.275621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:01.297411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f53db7506ce441d923fcaac902a34ab'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.346105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:01.358861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f85cf36366414b98ff5dcd728796f2', 'request_id': 'req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.364176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f85cf36366414b98ff5dcd728796f2', 'request_id': 'req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.372648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f85cf36366414b98ff5dcd728796f2', 'request_id': 'req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.376441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f53db7506ce441d923fcaac902a34ab'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.384971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f53db7506ce441d923fcaac902a34ab'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.385705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f53db7506ce441d923fcaac902a34ab'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.386340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f53db7506ce441d923fcaac902a34ab'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.411503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1041/2050] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:01 2022] DELETE /identity/v3/users/6f53db7506ce441d923fcaac902a34ab => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:01.412543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:01.414496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:01.434479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] The token cache is being invalidate because project 08f85cf36366414b98ff5dcd728796f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:01.434972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.435840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.438075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1010/2051] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:01 2022] DELETE /identity/v3/projects/08f85cf36366414b98ff5dcd728796f2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:01.442419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:01.444257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:01.544506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613fe1f-e541-4f16-8936-d6b21ef682e3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:01.553818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:01.554705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:01.555233 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:01.555642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] PATH_INFO: `/v3/projects/c912ce0dc0b842af95fe71ddea4e6e06` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:01.557358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c912ce0dc0b842af95fe71ddea4e6e06)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:01.559909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:01.566679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b89ee7-9be4-4c8d-be0b-3ca551beeeb0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:01.572731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:01.573401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:01.574569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:01.574846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] PATH_INFO: `/v3/users/6116146a462e412cb6dc51dea4b25696` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:01.576750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6116146a462e412cb6dc51dea4b25696)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:01.578641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:01.760663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6116146a462e412cb6dc51dea4b25696'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.766894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c912ce0dc0b842af95fe71ddea4e6e06', 'request_id': 'req-59b03cba-9b89-4f35-9db5-cb8966c255fd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.771588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6116146a462e412cb6dc51dea4b25696'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.846750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c912ce0dc0b842af95fe71ddea4e6e06', 'request_id': 'req-59b03cba-9b89-4f35-9db5-cb8966c255fd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.853122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c912ce0dc0b842af95fe71ddea4e6e06', 'request_id': 'req-59b03cba-9b89-4f35-9db5-cb8966c255fd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.864936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] The token cache is being invalidate because project c912ce0dc0b842af95fe71ddea4e6e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:01.865170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.865909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.867597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1042/2052] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:01 2022] DELETE /identity/v3/projects/c912ce0dc0b842af95fe71ddea4e6e06 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:01.868447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:01.869782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:01.932661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6116146a462e412cb6dc51dea4b25696'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.933438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6116146a462e412cb6dc51dea4b25696'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.948619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6116146a462e412cb6dc51dea4b25696'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:01.955878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1011/2053] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:01 2022] DELETE /identity/v3/users/6116146a462e412cb6dc51dea4b25696 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:01.958729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03cba-9b89-4f35-9db5-cb8966c255fd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:02.035017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:02.036646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.037083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.037527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] PATH_INFO: `/v3/users/570de5da908c4b31a8a22706642aafb4` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.039394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=570de5da908c4b31a8a22706642aafb4)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:02.041975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:02.062804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:02.065537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:02.088249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192932a5-6d6b-4fe3-85ba-7ebf17093900 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:02.093842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:02.094584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.095185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.095891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] PATH_INFO: `/v3/projects/b244817dce4241f7b1851f010f00030c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.098306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b244817dce4241f7b1851f010f00030c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:02.100297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:02.268963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b244817dce4241f7b1851f010f00030c', 'request_id': 'req-e0564313-8c50-48c2-bdd7-21844be7764a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.271982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570de5da908c4b31a8a22706642aafb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.277148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b244817dce4241f7b1851f010f00030c', 'request_id': 'req-e0564313-8c50-48c2-bdd7-21844be7764a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.283018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b244817dce4241f7b1851f010f00030c', 'request_id': 'req-e0564313-8c50-48c2-bdd7-21844be7764a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.349649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570de5da908c4b31a8a22706642aafb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.357681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570de5da908c4b31a8a22706642aafb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.358500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570de5da908c4b31a8a22706642aafb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.359130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-73f5c616-7a2b-4586-9417-c6a2b233a4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570de5da908c4b31a8a22706642aafb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.382892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1043/2054] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:01 2022] DELETE /identity/v3/users/570de5da908c4b31a8a22706642aafb4 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:02.409170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] The token cache is being invalidate because project b244817dce4241f7b1851f010f00030c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:02.409697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.411219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e0564313-8c50-48c2-bdd7-21844be7764a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.413647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1012/2055] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:02 2022] DELETE /identity/v3/projects/b244817dce4241f7b1851f010f00030c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:02.419223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab126a52-10a5-41d8-9ac2-f161455e4c72 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.419842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab126a52-10a5-41d8-9ac2-f161455e4c72 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.420140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab126a52-10a5-41d8-9ac2-f161455e4c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.446727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ab126a52-10a5-41d8-9ac2-f161455e4c72 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:02.446727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ab126a52-10a5-41d8-9ac2-f161455e4c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:02.520130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1044/2056] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:02.521872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c84e2-8dde-4882-88be-5875420d173f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.522198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c84e2-8dde-4882-88be-5875420d173f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.522480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c84e2-8dde-4882-88be-5875420d173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.545134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e40c84e2-8dde-4882-88be-5875420d173f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:02.546473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e40c84e2-8dde-4882-88be-5875420d173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:02.575618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1013/2057] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:02.576894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab126a52-10a5-41d8-9ac2-f161455e4c72 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:02.588277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab126a52-10a5-41d8-9ac2-f161455e4c72 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:02.594356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:02.594992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.595282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.595617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.596334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:02:02.596644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:02.598296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:02.610711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1045/2058] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:02:02.614850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c84e2-8dde-4882-88be-5875420d173f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:02.624853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c84e2-8dde-4882-88be-5875420d173f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:02.629638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:02.630292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.630580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.630858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.631552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:02.634097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:02.646017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:02.649586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:02.664564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4dedc-d991-4b2c-a85f-1b2d345660e1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:02.675045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:02.675045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.675045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.675045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] PATH_INFO: `/v3/projects/918875fab314446d89227411b5426197` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.675336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] RBAC: Authorizing `identity:delete_project(project_id=918875fab314446d89227411b5426197)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:02.675664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:02.734842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1014/2059] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:02 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:02.736644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:02.739589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:02.807253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918875fab314446d89227411b5426197', 'request_id': 'req-2a6e28cb-ea50-49cb-b541-af554d38476a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.829187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27693159-615f-4e8a-ab9f-8842fcf23510 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:02.901838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:02.902443 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.902724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.902999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] PATH_INFO: `/v3/projects/768f11935fcd4080b5045613f6b7cb28` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.904335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=768f11935fcd4080b5045613f6b7cb28)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:02.906245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:02.912391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918875fab314446d89227411b5426197', 'request_id': 'req-2a6e28cb-ea50-49cb-b541-af554d38476a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.924782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918875fab314446d89227411b5426197', 'request_id': 'req-2a6e28cb-ea50-49cb-b541-af554d38476a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.941551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] The token cache is being invalidate because project 918875fab314446d89227411b5426197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:02.941999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.942955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:02.949479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1046/2060] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:02 2022] DELETE /identity/v3/projects/918875fab314446d89227411b5426197 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:02.949479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:02.950208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:02.965215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e28cb-ea50-49cb-b541-af554d38476a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:02.973608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fb47a-c63a-4fd2-8baf-fc7538727525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:02.976595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fb47a-c63a-4fd2-8baf-fc7538727525 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:02.976595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fb47a-c63a-4fd2-8baf-fc7538727525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:02.976595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fb47a-c63a-4fd2-8baf-fc7538727525 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:02.976595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fb47a-c63a-4fd2-8baf-fc7538727525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:02.978044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fb47a-c63a-4fd2-8baf-fc7538727525 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.045785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768f11935fcd4080b5045613f6b7cb28', 'request_id': 'req-282b5263-7688-4526-954a-1216c8c52004'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:03.052138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768f11935fcd4080b5045613f6b7cb28', 'request_id': 'req-282b5263-7688-4526-954a-1216c8c52004'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:03.057823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768f11935fcd4080b5045613f6b7cb28', 'request_id': 'req-282b5263-7688-4526-954a-1216c8c52004'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:03.071959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1047/2061] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:02 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:03.071959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fd8f8-2b6c-4fc0-ac59-c9de91551866 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.071959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fd8f8-2b6c-4fc0-ac59-c9de91551866 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.071959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fd8f8-2b6c-4fc0-ac59-c9de91551866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.089270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fb4fd8f8-2b6c-4fc0-ac59-c9de91551866 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:03.090512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fb4fd8f8-2b6c-4fc0-ac59-c9de91551866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:03.136955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:03.149527 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] The token cache is being invalidate because project 768f11935fcd4080b5045613f6b7cb28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:03.149943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:03.150654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:03.155994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1015/2062] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:02 2022] DELETE /identity/v3/projects/768f11935fcd4080b5045613f6b7cb28 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:03.155994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.155994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:03.157916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1048/2063] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:03.159213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670f14e-8b22-47c0-91e7-599d39a9447a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.159506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670f14e-8b22-47c0-91e7-599d39a9447a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.159819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670f14e-8b22-47c0-91e7-599d39a9447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.176868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b5263-7688-4526-954a-1216c8c52004 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.181700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e670f14e-8b22-47c0-91e7-599d39a9447a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:03.183131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e670f14e-8b22-47c0-91e7-599d39a9447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:03.184388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f7bc3-829f-4a5d-b1fc-f86f0197096f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.185019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f7bc3-829f-4a5d-b1fc-f86f0197096f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.185308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f7bc3-829f-4a5d-b1fc-f86f0197096f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.185585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f7bc3-829f-4a5d-b1fc-f86f0197096f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.186314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f7bc3-829f-4a5d-b1fc-f86f0197096f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.187970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f7bc3-829f-4a5d-b1fc-f86f0197096f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.191229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1049/2064] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:03.192461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3cfe7-0586-4525-a484-b02eead41fa1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.195250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3cfe7-0586-4525-a484-b02eead41fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.195250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3cfe7-0586-4525-a484-b02eead41fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.201749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1016/2065] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:03.201749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845322a-75cd-4af6-b445-bf2885233b53 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.201749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845322a-75cd-4af6-b445-bf2885233b53 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.201749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845322a-75cd-4af6-b445-bf2885233b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.214157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b8f3cfe7-0586-4525-a484-b02eead41fa1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:03.215376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b8f3cfe7-0586-4525-a484-b02eead41fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:03.220586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6845322a-75cd-4af6-b445-bf2885233b53 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:03.221774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6845322a-75cd-4af6-b445-bf2885233b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:03.223993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1050/2066] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:03.224782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3cfe7-0586-4525-a484-b02eead41fa1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.230742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1017/2067] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:03.232257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845322a-75cd-4af6-b445-bf2885233b53 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.235842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3cfe7-0586-4525-a484-b02eead41fa1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.240875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.241246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845322a-75cd-4af6-b445-bf2885233b53 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.244593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.244593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.244593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.244593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.245403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.246937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.247681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.247972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.248255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.248963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:02:03.249277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.250940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.258473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1051/2068] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:03.259636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.264228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1018/2069] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:02:03.265563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.268115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:03.270389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f00ee7-e9fa-4686-b217-256bbcdd561c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.275224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.276093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.276382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.276666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.277370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:02:03.277707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.278348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfa31-fd09-480e-865b-762a3a2031e1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.279328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.283058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.283755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.284042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.284319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] PATH_INFO: `/v3/projects/c8f68a208091424cbafea8cccee4a38f/users/d8b0efb1f26f436dba7477eefd4e4054/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.291983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1052/2070] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:02:03.292788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.300290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f68a208091424cbafea8cccee4a38f, user_id=d8b0efb1f26f436dba7477eefd4e4054, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.303141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.305581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f74ca-f4cb-4964-96fb-8e74e77e75e1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.310574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f0824-dee5-4208-85aa-bcf6702fea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.311182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f0824-dee5-4208-85aa-bcf6702fea8d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.311470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f0824-dee5-4208-85aa-bcf6702fea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.311785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f0824-dee5-4208-85aa-bcf6702fea8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.312486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f0824-dee5-4208-85aa-bcf6702fea8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.314439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f0824-dee5-4208-85aa-bcf6702fea8d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.378706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1019/2071] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:03 2022] PUT /identity/v3/projects/c8f68a208091424cbafea8cccee4a38f/users/d8b0efb1f26f436dba7477eefd4e4054/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:03.379792 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.419249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1053/2072] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:03.420435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604adbb-732e-425f-90d6-e30d8eff6f5f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.421347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604adbb-732e-425f-90d6-e30d8eff6f5f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.421347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604adbb-732e-425f-90d6-e30d8eff6f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.453251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5604adbb-732e-425f-90d6-e30d8eff6f5f None None] MFA Rules not processed for user `d8b0efb1f26f436dba7477eefd4e4054`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:03.454777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5604adbb-732e-425f-90d6-e30d8eff6f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:03.459116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eda738-9326-44ef-8014-da46d7c1c329 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.517362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.517987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.518271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.518545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.519404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.521225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.525607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1054/2073] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:03.525607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604adbb-732e-425f-90d6-e30d8eff6f5f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.534480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604adbb-732e-425f-90d6-e30d8eff6f5f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.540048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.540673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.540986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.541263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.541959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.544263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.635929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1020/2074] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/projects => generated 387 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:03.637941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.669846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1055/2075] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:03.669846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.678734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63425e67-1479-45c0-89fb-521e9e34fc87 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.722017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4e04-fe0e-4d60-a26a-f9152cad197d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.733003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.733997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.734624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.735024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.736242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.739159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.744955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.745892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.747679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.747679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.747679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.750057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.864271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1056/2076] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:03.865066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.870506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1021/2077] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:03.871638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.878888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb2747-c463-4fdc-8862-c9b17041574a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.937560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c224df-6276-4116-80a7-a50644e1d7c5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.938049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.938679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.938977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.939244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.940028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.941610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.942587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.943209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.943500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.943807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.945009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.946578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.952037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1057/2078] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:03.952824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.960032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1022/2079] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:03.961498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:03.963796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5b75-eb87-4ea8-9ff6-3fe68d6442f1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.970162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.971016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.971420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.971831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.972130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761befa-9150-4171-a459-ea55d4d56099 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:03.972762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.974971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.976818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:03.977446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:03.977712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:03.978019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:03.978670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:03.980293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:03.990657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1023/2080] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:03.991497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.001024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecad30-c157-43f3-8d8a-34d1eaf26fe6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.005730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.006320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.006601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.006875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.008337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.009945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.020234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1024/2081] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:04.021208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.031277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c86b96-4733-452f-be3f-0b7b5d571ad3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.036002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.036634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.036919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.037197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] PATH_INFO: `/v3/projects/b2a874e651ab429f9544d9247a0aa8aa/users/171e3adfcd8d40b9b593b7c6b426ad82/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.052822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a874e651ab429f9544d9247a0aa8aa, user_id=171e3adfcd8d40b9b593b7c6b426ad82, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.055688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.074086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1058/2082] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:03 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:04.074086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.082021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462945c5-1274-4dac-aaa8-87ca89e81cf8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.087797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d2e06-1c91-402d-bbf9-93c2024f5a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.088468 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2e06-1c91-402d-bbf9-93c2024f5a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.088767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2e06-1c91-402d-bbf9-93c2024f5a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.089047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2e06-1c91-402d-bbf9-93c2024f5a92 admin admin] PATH_INFO: `/v3/projects/3129476636bb43e9a14ef674c4720233/users/14e5f1b4073742b4b9bd8a48cce38341/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.104960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d2e06-1c91-402d-bbf9-93c2024f5a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3129476636bb43e9a14ef674c4720233, user_id=14e5f1b4073742b4b9bd8a48cce38341, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.107793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d2e06-1c91-402d-bbf9-93c2024f5a92 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.133740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1025/2083] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:04 2022] PUT /identity/v3/projects/b2a874e651ab429f9544d9247a0aa8aa/users/171e3adfcd8d40b9b593b7c6b426ad82/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:04.137026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.185175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1059/2084] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:04 2022] PUT /identity/v3/projects/3129476636bb43e9a14ef674c4720233/users/14e5f1b4073742b4b9bd8a48cce38341/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:04.186531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bccfbb-cc3e-4314-9714-c6cdfa347ecf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.186829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bccfbb-cc3e-4314-9714-c6cdfa347ecf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.188182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bccfbb-cc3e-4314-9714-c6cdfa347ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.211719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efced7c2-e0ea-4e7c-a6d2-fba830873ead admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.227297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-86bccfbb-cc3e-4314-9714-c6cdfa347ecf None None] MFA Rules not processed for user `171e3adfcd8d40b9b593b7c6b426ad82`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:04.228679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-86bccfbb-cc3e-4314-9714-c6cdfa347ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:04.267323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23953b8-d3c6-40c4-8e4f-636c253ae622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.267980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23953b8-d3c6-40c4-8e4f-636c253ae622 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.268268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23953b8-d3c6-40c4-8e4f-636c253ae622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.268546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23953b8-d3c6-40c4-8e4f-636c253ae622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.269240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23953b8-d3c6-40c4-8e4f-636c253ae622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.270839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23953b8-d3c6-40c4-8e4f-636c253ae622 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.280948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1026/2085] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:04.282166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.282457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.282739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.298004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1060/2086] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:04.298973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bccfbb-cc3e-4314-9714-c6cdfa347ecf None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.302992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:04.306794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:04.310993 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bccfbb-cc3e-4314-9714-c6cdfa347ecf None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.313349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1027/2087] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:04.314714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.316129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.316262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:04.316929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.317479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.317648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] PATH_INFO: `/v3/projects/a24625e4832444249415b6bb14a88987/users/ab40eb3f7936411dbde9c8151e8d76b6/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.334816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24625e4832444249415b6bb14a88987, user_id=ab40eb3f7936411dbde9c8151e8d76b6, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.337698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.355503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:04.372923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908fe91-21e3-43fc-9b49-f628e2d9e80f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.376654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.377265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.377572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.377872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.387008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.388615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.397689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1028/2088] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:02:04.398591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.399975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:04.436688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1061/2089] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:04 2022] PUT /identity/v3/projects/a24625e4832444249415b6bb14a88987/users/ab40eb3f7936411dbde9c8151e8d76b6/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:04.439953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:04.440646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.463034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac330c-f6f9-48a3-b203-5638578bbf61 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.516702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197be19-ac5b-41f6-8bc1-e28d54597a83 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.518996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abe2f2-8e93-4fe2-a7d7-706773f04d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.519701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe2f2-8e93-4fe2-a7d7-706773f04d88 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.519989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe2f2-8e93-4fe2-a7d7-706773f04d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.520305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe2f2-8e93-4fe2-a7d7-706773f04d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.522205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.523057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.523425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.523910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.524624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.526336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.530248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abe2f2-8e93-4fe2-a7d7-706773f04d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.532437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abe2f2-8e93-4fe2-a7d7-706773f04d88 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.537346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1062/2090] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:02:04.542130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.551906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1029/2091] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:04.552422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5277e-15ca-400f-8116-391c51e626e0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.557935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.558608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.558898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.559180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] PATH_INFO: `/v3/projects/a24625e4832444249415b6bb14a88987/users/ab40eb3f7936411dbde9c8151e8d76b6/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.561215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24625e4832444249415b6bb14a88987, user_id=ab40eb3f7936411dbde9c8151e8d76b6, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.563955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.574953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1063/2092] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:04 2022] PUT /identity/v3/projects/a24625e4832444249415b6bb14a88987/users/ab40eb3f7936411dbde9c8151e8d76b6/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:04.586267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae4ff0-50cc-4ad9-b227-e8a5c8fecafe None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.586600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae4ff0-50cc-4ad9-b227-e8a5c8fecafe None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.586909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae4ff0-50cc-4ad9-b227-e8a5c8fecafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.615207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d7ae4ff0-50cc-4ad9-b227-e8a5c8fecafe None None] MFA Rules not processed for user `ab40eb3f7936411dbde9c8151e8d76b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:04.617257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d7ae4ff0-50cc-4ad9-b227-e8a5c8fecafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:04.687514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1030/2093] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:04.698490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.700262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:04.769262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ac74c-c521-4800-918b-2e9df4cfec00 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.773400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.774035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.774534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.774711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:04.784621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:04.786084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:04.796910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1064/2094] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:04.933428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae4ff0-50cc-4ad9-b227-e8a5c8fecafe None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:04.943848 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae4ff0-50cc-4ad9-b227-e8a5c8fecafe None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:04.947677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da18428-62bf-4757-8fa8-c2453ca16000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:04.948297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18428-62bf-4757-8fa8-c2453ca16000 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:04.948578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18428-62bf-4757-8fa8-c2453ca16000 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:04.948869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da18428-62bf-4757-8fa8-c2453ca16000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:05.008533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da18428-62bf-4757-8fa8-c2453ca16000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:05.010013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da18428-62bf-4757-8fa8-c2453ca16000 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:05.019262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1031/2095] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:05.115345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:05.188573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:05.919924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:06.109664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:07.298764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:07.620172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:09.426961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:09.517971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210492c-6e13-4080-923f-543a655ece83 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:09.524242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:09.525189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:09.525302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:09.525631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] PATH_INFO: `/v3/users/d8b0efb1f26f436dba7477eefd4e4054` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:09.527002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b0efb1f26f436dba7477eefd4e4054)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:09.529118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:09.769163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b0efb1f26f436dba7477eefd4e4054'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:09.840506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b0efb1f26f436dba7477eefd4e4054'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:09.848236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b0efb1f26f436dba7477eefd4e4054'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:09.848911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b0efb1f26f436dba7477eefd4e4054'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:09.849685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b0efb1f26f436dba7477eefd4e4054'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:09.878396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1065/2096] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:09 2022] DELETE /identity/v3/users/d8b0efb1f26f436dba7477eefd4e4054 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:09.880100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c8fe6-509f-4066-b995-109617921b41 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:09.880551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c8fe6-509f-4066-b995-109617921b41 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:09.880666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c8fe6-509f-4066-b995-109617921b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:09.901759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-108c8fe6-509f-4066-b995-109617921b41 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:09.902550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-108c8fe6-509f-4066-b995-109617921b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:09.964246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:09.979301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1032/2097] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:09.991877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:09.993707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:10.035214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:10.067294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b9819-8fcc-490a-bc60-2c7c370480bc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:10.071647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a93874-5a28-4ef5-98f7-e4aa5de2c5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:10.072287 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a93874-5a28-4ef5-98f7-e4aa5de2c5cf service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:10.072572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a93874-5a28-4ef5-98f7-e4aa5de2c5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:10.072848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a93874-5a28-4ef5-98f7-e4aa5de2c5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:10.085999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a93874-5a28-4ef5-98f7-e4aa5de2c5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:10.087483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a93874-5a28-4ef5-98f7-e4aa5de2c5cf service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:10.104782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1066/2098] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:02:10.704867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c8fe6-509f-4066-b995-109617921b41 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:10.709710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-108c8fe6-509f-4066-b995-109617921b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:10.785308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c8fe6-509f-4066-b995-109617921b41 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:10.793021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:10.793816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:10.794169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:10.794522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] PATH_INFO: `/v3/projects/c8f68a208091424cbafea8cccee4a38f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:10.795929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f68a208091424cbafea8cccee4a38f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:10.797961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:10.978555 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:10.979083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:10.979496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:10.985227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f68a208091424cbafea8cccee4a38f', 'request_id': 'req-21f12799-ea06-407c-8cd0-34a41b6f41b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.003749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f68a208091424cbafea8cccee4a38f', 'request_id': 'req-21f12799-ea06-407c-8cd0-34a41b6f41b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.019882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f68a208091424cbafea8cccee4a38f', 'request_id': 'req-21f12799-ea06-407c-8cd0-34a41b6f41b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.025439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] MFA Rules not processed for user `14e5f1b4073742b4b9bd8a48cce38341`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:11.026764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:11.095424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] The token cache is being invalidate because project c8f68a208091424cbafea8cccee4a38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:11.095866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.096814 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.098416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1033/2099] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:10 2022] DELETE /identity/v3/projects/c8f68a208091424cbafea8cccee4a38f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:11.103245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:11.104930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:11.113300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1067/2100] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:02:11.128387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:11.133640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:11.206909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f12799-ea06-407c-8cd0-34a41b6f41b7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:11.213986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:11.214867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:11.215148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:11.215802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] PATH_INFO: `/v3/users/ab40eb3f7936411dbde9c8151e8d76b6` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:11.217100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab40eb3f7936411dbde9c8151e8d76b6)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:11.219125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:11.256956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608afe7-babd-406d-bfe5-ae986105feaf None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:11.264370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:11.265094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:11.265522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:11.265873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:11.269013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:11.281574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:11.283135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:11.293531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1068/2101] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:11.438777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab40eb3f7936411dbde9c8151e8d76b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.446729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab40eb3f7936411dbde9c8151e8d76b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.653334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab40eb3f7936411dbde9c8151e8d76b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.654118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab40eb3f7936411dbde9c8151e8d76b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.690587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e38f25cb-8684-4b8e-946d-fc41fedfc1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab40eb3f7936411dbde9c8151e8d76b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:11.718129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1034/2102] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:11 2022] DELETE /identity/v3/users/ab40eb3f7936411dbde9c8151e8d76b6 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:12.208727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:12.232016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba92248-bbfe-4236-be21-c97dc7aeef8e service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:12.323151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:12.335896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:12.336545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:12.336831 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:12.337113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] PATH_INFO: `/v3/projects/a24625e4832444249415b6bb14a88987` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:12.338273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24625e4832444249415b6bb14a88987)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:12.340257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:12.507921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24625e4832444249415b6bb14a88987', 'request_id': 'req-5b77b2f2-80eb-4e7c-99c2-52c14908f420'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:12.580504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24625e4832444249415b6bb14a88987', 'request_id': 'req-5b77b2f2-80eb-4e7c-99c2-52c14908f420'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:12.582437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24625e4832444249415b6bb14a88987', 'request_id': 'req-5b77b2f2-80eb-4e7c-99c2-52c14908f420'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:12.594294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] The token cache is being invalidate because project a24625e4832444249415b6bb14a88987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:12.594584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:12.595264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5b77b2f2-80eb-4e7c-99c2-52c14908f420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:12.596806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1069/2103] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:12 2022] DELETE /identity/v3/projects/a24625e4832444249415b6bb14a88987 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:12.602405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954896b-9a79-40ba-b366-965906e4d953 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:12.602905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954896b-9a79-40ba-b366-965906e4d953 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:12.603011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954896b-9a79-40ba-b366-965906e4d953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:12.610906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:12.623590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3954896b-9a79-40ba-b366-965906e4d953 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:12.624949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3954896b-9a79-40ba-b366-965906e4d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:12.691078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1035/2104] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:12.704666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdef9b-bec9-4c6e-b6da-999878efd91d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:12.705282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdef9b-bec9-4c6e-b6da-999878efd91d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:12.705401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdef9b-bec9-4c6e-b6da-999878efd91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:12.730563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-aabdef9b-bec9-4c6e-b6da-999878efd91d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:12.732091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aabdef9b-bec9-4c6e-b6da-999878efd91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:12.744140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1070/2105] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:12.748885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954896b-9a79-40ba-b366-965906e4d953 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:12.762631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954896b-9a79-40ba-b366-965906e4d953 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:12.771751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:12.773224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:12.776208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:12.776208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:12.776208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:02:12.776208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:12.777280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:12.789889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1036/2106] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:02:12.794579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdef9b-bec9-4c6e-b6da-999878efd91d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:12.805569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdef9b-bec9-4c6e-b6da-999878efd91d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:12.810737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:12.811649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:12.811772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:12.812007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:12.812671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:12.814577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:12.911640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1071/2107] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:12 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:12.916118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:12.931014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b762e-01f3-41c4-b941-159f0a33c284 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:13.007209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:13.008208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:13.008648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:13.009074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:13.009949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:13.012234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:13.172202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1037/2108] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:12 2022] POST /identity/v3/users => generated 290 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:13.176735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:13.199035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ce700-5e86-4d7f-bbf8-32b09e4b3d82 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:13.209356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:13.210638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:13.210907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:13.211191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:13.211989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:13.213802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:13.227028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1072/2109] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:13.230805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:13.242740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92b44f-21cb-4a01-8a9a-1aa979228d35 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:13.248465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832e139-14cd-4e2c-a139-8a65da518785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:13.249351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832e139-14cd-4e2c-a139-8a65da518785 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:13.249746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832e139-14cd-4e2c-a139-8a65da518785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:13.250416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832e139-14cd-4e2c-a139-8a65da518785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:13.251324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832e139-14cd-4e2c-a139-8a65da518785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:13.253552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832e139-14cd-4e2c-a139-8a65da518785 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:13.271712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1038/2110] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:13.274745 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:13.286787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad3422-c22b-48a0-8fdf-052bfebdc266 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:13.293892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:13.294983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:13.295422 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:13.296105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] PATH_INFO: `/v3/projects/2caf1f473bbe40f68d9597f782e4d1dd/users/39c9f2b686ec4cf6a769fd1c1d59aa2e/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:13.317864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2caf1f473bbe40f68d9597f782e4d1dd, user_id=39c9f2b686ec4cf6a769fd1c1d59aa2e, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:13.321730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:13.409391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1073/2111] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:13 2022] PUT /identity/v3/projects/2caf1f473bbe40f68d9597f782e4d1dd/users/39c9f2b686ec4cf6a769fd1c1d59aa2e/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:13.421740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef0450-c5b7-4a8f-8d15-a6ef007c6c67 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:13.422252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef0450-c5b7-4a8f-8d15-a6ef007c6c67 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:13.422860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef0450-c5b7-4a8f-8d15-a6ef007c6c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:13.462986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-60ef0450-c5b7-4a8f-8d15-a6ef007c6c67 None None] MFA Rules not processed for user `39c9f2b686ec4cf6a769fd1c1d59aa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:13.464270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-60ef0450-c5b7-4a8f-8d15-a6ef007c6c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:13.552038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1039/2112] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:13.556711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:13.652816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13ff78-f342-4035-b243-9462444af9ab admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:13.659073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:13.659744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:13.660030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:13.660309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:13.661001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:13.663080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:13.732986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:13.743466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1074/2113] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:13 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:02:13.747669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef0450-c5b7-4a8f-8d15-a6ef007c6c67 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:13.761398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef0450-c5b7-4a8f-8d15-a6ef007c6c67 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:13.830297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:13.831091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:13.831509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:13.831904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:13.832729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:13.835712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:14.000685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1040/2114] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:13 2022] POST /identity/v3/users => generated 296 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:02:14.005155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:14.016878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256919f-da04-4473-b969-61ff17138f19 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:14.022379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:14.023013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:14.023301 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:14.023617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:14.024360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:14.026028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:14.038465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1075/2115] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:14.043225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:14.065578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5732e96-4cde-43a5-8aa7-5155ebc9e08a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:14.072395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:14.073206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:14.073498 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:14.073781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] PATH_INFO: `/v3/projects/ddae9d7e7a2249179ada4694f296a75d/users/ca7bd84f20f0417b8654c9d5fd351e1c/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:14.122344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddae9d7e7a2249179ada4694f296a75d, user_id=ca7bd84f20f0417b8654c9d5fd351e1c, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:14.138072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:14.224554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1041/2116] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:14 2022] PUT /identity/v3/projects/ddae9d7e7a2249179ada4694f296a75d/users/ca7bd84f20f0417b8654c9d5fd351e1c/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:14.228823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:14.242873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88294c98-2743-413d-8cf4-7d2785656e21 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:14.328722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd81202-b180-445c-8153-594ec155cf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:14.329673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd81202-b180-445c-8153-594ec155cf79 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:14.330160 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd81202-b180-445c-8153-594ec155cf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:14.330822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd81202-b180-445c-8153-594ec155cf79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:14.331694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd81202-b180-445c-8153-594ec155cf79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:14.333524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd81202-b180-445c-8153-594ec155cf79 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:14.344456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1076/2117] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:14.349003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:14.362166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de9376-5550-4117-a835-a9dc7cf72c67 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:14.368485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:14.369442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:14.370003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:14.370457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] PATH_INFO: `/v3/projects/ddae9d7e7a2249179ada4694f296a75d/users/ca7bd84f20f0417b8654c9d5fd351e1c/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:14.372814 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddae9d7e7a2249179ada4694f296a75d, user_id=ca7bd84f20f0417b8654c9d5fd351e1c, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:14.377718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:14.391812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1042/2118] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:14 2022] PUT /identity/v3/projects/ddae9d7e7a2249179ada4694f296a75d/users/ca7bd84f20f0417b8654c9d5fd351e1c/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:14.410117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5b1ce-34e5-47f8-9fb2-7eda892be40b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:14.410117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5b1ce-34e5-47f8-9fb2-7eda892be40b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:14.410117 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5b1ce-34e5-47f8-9fb2-7eda892be40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:14.452551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a8d5b1ce-34e5-47f8-9fb2-7eda892be40b None None] MFA Rules not processed for user `ca7bd84f20f0417b8654c9d5fd351e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:14.454059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a8d5b1ce-34e5-47f8-9fb2-7eda892be40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:14.560442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1077/2119] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:14.586736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:14.588508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:14.652197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3aef5-3741-4144-93cc-095b90d3df21 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:14.656393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:14.657042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:14.657332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:14.657615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:14.669064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:14.670520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:14.682528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1043/2120] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:14.813758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2a6ed-e4fe-4787-9097-d223a5a273e4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:14.814105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2a6ed-e4fe-4787-9097-d223a5a273e4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:14.814397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2a6ed-e4fe-4787-9097-d223a5a273e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:14.836915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-15c2a6ed-e4fe-4787-9097-d223a5a273e4 None None] MFA Rules not processed for user `14e5f1b4073742b4b9bd8a48cce38341`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:14.838211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-15c2a6ed-e4fe-4787-9097-d223a5a273e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:14.910956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1078/2121] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:14.972404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:15.415916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:15.493408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcb3b4-c825-44fa-ae77-268376fb477b service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:15.500221 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:15.500647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:15.500926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:15.501226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] PATH_INFO: `/v3/users/39c9f2b686ec4cf6a769fd1c1d59aa2e` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:15.504095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39c9f2b686ec4cf6a769fd1c1d59aa2e)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:15.506020 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:15.566590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:15.712065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39c9f2b686ec4cf6a769fd1c1d59aa2e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:15.719081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39c9f2b686ec4cf6a769fd1c1d59aa2e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:15.810077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39c9f2b686ec4cf6a769fd1c1d59aa2e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:15.812769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39c9f2b686ec4cf6a769fd1c1d59aa2e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:15.828511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39c9f2b686ec4cf6a769fd1c1d59aa2e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:15.839665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1044/2122] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:15 2022] DELETE /identity/v3/users/39c9f2b686ec4cf6a769fd1c1d59aa2e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:15.844217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:15.844701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:15.844806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:15.865659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:15.882427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:15.894101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:15.963644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1079/2123] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:15.973432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:15.976514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:16.044664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001614b-fcf5-41f3-83d3-a1e579ce3f4b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:16.048264 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:16.048897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:16.049178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:16.049452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:16.059047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:16.060516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:16.070859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1045/2124] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:16.539370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:16.541316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:16.560152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d4ae8-b223-448c-b535-c0d4025b7acd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:16.566828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:16.567472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:16.567799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:16.568172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] PATH_INFO: `/v3/projects/2caf1f473bbe40f68d9597f782e4d1dd` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:16.569682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2caf1f473bbe40f68d9597f782e4d1dd)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:16.571977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:16.716547 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2caf1f473bbe40f68d9597f782e4d1dd', 'request_id': 'req-10e9330c-dc36-4717-9f97-f82fba9e9346'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:16.789981 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2caf1f473bbe40f68d9597f782e4d1dd', 'request_id': 'req-10e9330c-dc36-4717-9f97-f82fba9e9346'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:16.799180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2caf1f473bbe40f68d9597f782e4d1dd', 'request_id': 'req-10e9330c-dc36-4717-9f97-f82fba9e9346'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:16.840481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] The token cache is being invalidate because project 2caf1f473bbe40f68d9597f782e4d1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:16.840779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:16.841480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:16.848355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1080/2125] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:16 2022] DELETE /identity/v3/projects/2caf1f473bbe40f68d9597f782e4d1dd => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:02:16.850329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:16.867160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b5a81-1a68-46a7-89b7-5b574e979338 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:16.939861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:16.940598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:16.941040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:16.941431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] PATH_INFO: `/v3/users/ca7bd84f20f0417b8654c9d5fd351e1c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:16.942982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca7bd84f20f0417b8654c9d5fd351e1c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:16.946229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:16.986540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:17.214052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca7bd84f20f0417b8654c9d5fd351e1c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:17.220884 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca7bd84f20f0417b8654c9d5fd351e1c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:17.320809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca7bd84f20f0417b8654c9d5fd351e1c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:17.321510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca7bd84f20f0417b8654c9d5fd351e1c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:17.335209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-019ffdd1-34ee-4639-a3dc-803167415c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca7bd84f20f0417b8654c9d5fd351e1c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:17.342025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1046/2126] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:16 2022] DELETE /identity/v3/users/ca7bd84f20f0417b8654c9d5fd351e1c => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:17.797252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:17.798793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:17.813570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9330c-dc36-4717-9f97-f82fba9e9346 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:17.876727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:17.877405 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:17.877655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:17.877930 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] PATH_INFO: `/v3/projects/ddae9d7e7a2249179ada4694f296a75d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:17.879104 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddae9d7e7a2249179ada4694f296a75d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:17.881891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:17.988139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:17.995392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddae9d7e7a2249179ada4694f296a75d', 'request_id': 'req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:18.108866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddae9d7e7a2249179ada4694f296a75d', 'request_id': 'req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:18.117225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddae9d7e7a2249179ada4694f296a75d', 'request_id': 'req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:18.153199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] The token cache is being invalidate because project ddae9d7e7a2249179ada4694f296a75d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:18.153199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:18.153199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4de445e-8c4b-4e36-a9b0-b7e8c7cbdfbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:18.153199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1081/2127] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:17 2022] DELETE /identity/v3/projects/ddae9d7e7a2249179ada4694f296a75d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:18.156930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475e9c-7b2e-4f9a-b0b8-a0b8e7301f99 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.157426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475e9c-7b2e-4f9a-b0b8-a0b8e7301f99 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.157615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475e9c-7b2e-4f9a-b0b8-a0b8e7301f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.182242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-94475e9c-7b2e-4f9a-b0b8-a0b8e7301f99 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:18.183495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-94475e9c-7b2e-4f9a-b0b8-a0b8e7301f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:18.264073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1047/2128] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:18.266370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5cc88-188f-4025-ba96-dae9c0b95eaa None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.266370 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5cc88-188f-4025-ba96-dae9c0b95eaa None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.266753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5cc88-188f-4025-ba96-dae9c0b95eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.296010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a3b5cc88-188f-4025-ba96-dae9c0b95eaa None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:18.297162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a3b5cc88-188f-4025-ba96-dae9c0b95eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:18.306055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1082/2129] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:02:18.310330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475e9c-7b2e-4f9a-b0b8-a0b8e7301f99 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:18.320570 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475e9c-7b2e-4f9a-b0b8-a0b8e7301f99 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:18.325275 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:18.325893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.326179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.326458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.327158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:02:18.327467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:18.329162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:18.340906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1048/2130] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:02:18.345221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5cc88-188f-4025-ba96-dae9c0b95eaa None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:18.355172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5cc88-188f-4025-ba96-dae9c0b95eaa None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:18.360164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:18.360794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.361099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.361404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.362087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:18.364095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:18.482882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1083/2131] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:18 2022] POST /identity/v3/projects => generated 399 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:18.486314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:18.496609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004b1bf-7d6c-43fc-87b5-6395063604bd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:18.557401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:18.557401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.557401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.557401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.557815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:18.557815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:18.675147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1049/2132] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:18 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:18.680516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:18.691036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76761d7d-3669-42ec-b6ff-05017ff0c78e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:18.695897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:18.696509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.696816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.697094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.697791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:18.699391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:18.709955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1084/2133] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:18.713751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:18.723674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae93c4-5c7e-4e73-9bad-15ac90912879 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:18.729143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a4b6b-ec20-4211-8d80-06b0781f5bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:18.729855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a4b6b-ec20-4211-8d80-06b0781f5bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.730049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a4b6b-ec20-4211-8d80-06b0781f5bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.730329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a4b6b-ec20-4211-8d80-06b0781f5bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.731147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a4b6b-ec20-4211-8d80-06b0781f5bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:18.732796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a4b6b-ec20-4211-8d80-06b0781f5bfb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:18.738467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:18.744244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1050/2134] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:18.748505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:18.760096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7c69b-29e3-4152-8ef2-21edd89122e7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:18.765694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:18.771096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.771096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.771096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] PATH_INFO: `/v3/projects/a94f3a5f90f04683803460e9305ef0c4/users/897acf18ad3e46b88123dd65e9fac130/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.782546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94f3a5f90f04683803460e9305ef0c4, user_id=897acf18ad3e46b88123dd65e9fac130, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:18.785269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:18.857092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1085/2135] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:18 2022] PUT /identity/v3/projects/a94f3a5f90f04683803460e9305ef0c4/users/897acf18ad3e46b88123dd65e9fac130/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:18.867569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e1ba1-b94a-456b-8ddd-9a46f3c9f4be None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:18.867881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e1ba1-b94a-456b-8ddd-9a46f3c9f4be None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:18.868171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e1ba1-b94a-456b-8ddd-9a46f3c9f4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:18.897352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e18e1ba1-b94a-456b-8ddd-9a46f3c9f4be None None] MFA Rules not processed for user `897acf18ad3e46b88123dd65e9fac130`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:18.898588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e18e1ba1-b94a-456b-8ddd-9a46f3c9f4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:18.962439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1051/2136] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:18.973789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:18.975652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:19.033224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f94da-9134-46e1-86b4-b18d57dce564 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:19.036992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:19.037802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:19.038093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:19.038374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:19.046894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:19.048407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:19.057157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1086/2137] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:29.049314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=32 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:29.053165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:35.565012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5dd11-d8de-4859-9e4d-a731fb0c34e9 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:35.565485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5dd11-d8de-4859-9e4d-a731fb0c34e9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:35.565600 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5dd11-d8de-4859-9e4d-a731fb0c34e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:35.585032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-06d5dd11-d8de-4859-9e4d-a731fb0c34e9 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:35.586237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-06d5dd11-d8de-4859-9e4d-a731fb0c34e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:35.641172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1052/2138] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:02:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:35.684803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:35.686247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:35.756768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625ae87-2f37-40f4-87de-77689480568b service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:35.762842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0871f5c-64d2-4bf5-9b02-240dc1a294de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:35.763675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0871f5c-64d2-4bf5-9b02-240dc1a294de service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:35.763957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0871f5c-64d2-4bf5-9b02-240dc1a294de service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:35.764289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0871f5c-64d2-4bf5-9b02-240dc1a294de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:35.773550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0871f5c-64d2-4bf5-9b02-240dc1a294de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:35.775126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0871f5c-64d2-4bf5-9b02-240dc1a294de service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:35.783132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1087/2139] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:02:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:02:36.475564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5dd11-d8de-4859-9e4d-a731fb0c34e9 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:36.568194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5dd11-d8de-4859-9e4d-a731fb0c34e9 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:36.575122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:36.576660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:36.576660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:36.577035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] PATH_INFO: `/v3/users/897acf18ad3e46b88123dd65e9fac130` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:36.578290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=897acf18ad3e46b88123dd65e9fac130)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:36.580429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:36.825093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '897acf18ad3e46b88123dd65e9fac130'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:36.835144 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '897acf18ad3e46b88123dd65e9fac130'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:36.906147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '897acf18ad3e46b88123dd65e9fac130'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:36.906860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '897acf18ad3e46b88123dd65e9fac130'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:36.922735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '897acf18ad3e46b88123dd65e9fac130'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:36.936402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1053/2140] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:02:36 2022] DELETE /identity/v3/users/897acf18ad3e46b88123dd65e9fac130 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:36.941692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:36.942261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:36.942602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:36.971613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:36.974333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:37.070826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1088/2141] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:37.080761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:37.082479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:37.116168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:37.152953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df1493-a755-4518-9f8d-ef9f05397312 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:37.157276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385a341-cf88-466f-96d5-ce99f77f2707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:37.157975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385a341-cf88-466f-96d5-ce99f77f2707 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:37.158325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385a341-cf88-466f-96d5-ce99f77f2707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:37.158787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385a341-cf88-466f-96d5-ce99f77f2707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:37.180586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385a341-cf88-466f-96d5-ce99f77f2707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:37.182106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385a341-cf88-466f-96d5-ce99f77f2707 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:37.194059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1054/2142] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:37.749568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:37.752198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:37.773460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7bc6f-e943-4b9c-83f6-5cb0b861f88d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:37.783071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:37.784495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:37.784847 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:37.785292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] PATH_INFO: `/v3/projects/a94f3a5f90f04683803460e9305ef0c4` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:37.787193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94f3a5f90f04683803460e9305ef0c4)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:37.790639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:37.971490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94f3a5f90f04683803460e9305ef0c4', 'request_id': 'req-895bc760-2cb4-4882-8c3a-0846d501ce52'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:38.100970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94f3a5f90f04683803460e9305ef0c4', 'request_id': 'req-895bc760-2cb4-4882-8c3a-0846d501ce52'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:38.107746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94f3a5f90f04683803460e9305ef0c4', 'request_id': 'req-895bc760-2cb4-4882-8c3a-0846d501ce52'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:38.121018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] The token cache is being invalidate because project a94f3a5f90f04683803460e9305ef0c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:02:38.121324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:38.122131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-895bc760-2cb4-4882-8c3a-0846d501ce52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:02:38.123889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1089/2143] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:02:37 2022] DELETE /identity/v3/projects/a94f3a5f90f04683803460e9305ef0c4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:38.130530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672deb5-d2a4-49a4-b475-bc7ad9b7dae8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.131069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672deb5-d2a4-49a4-b475-bc7ad9b7dae8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.131246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672deb5-d2a4-49a4-b475-bc7ad9b7dae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.154094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2672deb5-d2a4-49a4-b475-bc7ad9b7dae8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:38.155308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2672deb5-d2a4-49a4-b475-bc7ad9b7dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:38.244740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1055/2144] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:38.254009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550efb4e-1899-4c12-9035-dbf60ccd9827 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.254534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550efb4e-1899-4c12-9035-dbf60ccd9827 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.255803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550efb4e-1899-4c12-9035-dbf60ccd9827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.285123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-550efb4e-1899-4c12-9035-dbf60ccd9827 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:38.287375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-550efb4e-1899-4c12-9035-dbf60ccd9827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:38.299586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1090/2145] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:38.305146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672deb5-d2a4-49a4-b475-bc7ad9b7dae8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:38.319238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672deb5-d2a4-49a4-b475-bc7ad9b7dae8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:38.325046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:38.325819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.326323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.326702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.327588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:02:38.328198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:38.330642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:38.351716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1056/2146] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:02:38.355342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550efb4e-1899-4c12-9035-dbf60ccd9827 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:38.367356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550efb4e-1899-4c12-9035-dbf60ccd9827 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:38.372306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:38.372972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.373300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.373583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.374400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:38.376577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:38.381946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:38.488565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1091/2147] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:38 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:02:38.496317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:38.507378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92c1e9-42d0-4d4c-8981-1fea3eb7731b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:38.576408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:38.577258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.577649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.578672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.579473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:38.585378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:38.681657 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1057/2148] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:38 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:38.685599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:38.697594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58307e61-bebc-4125-be6e-a6465d0147ba admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:38.702465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:38.703104 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.703387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.703711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.704488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:38.707052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:38.723784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:38.725145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1092/2149] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:38.735746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b78bd-138b-4504-af05-f4824ec9efd7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:38.742016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2eee3-fb25-40f9-a345-f3f3f8377582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:38.743213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2eee3-fb25-40f9-a345-f3f3f8377582 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.743502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2eee3-fb25-40f9-a345-f3f3f8377582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.744071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2eee3-fb25-40f9-a345-f3f3f8377582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.755653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2eee3-fb25-40f9-a345-f3f3f8377582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:38.758867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2eee3-fb25-40f9-a345-f3f3f8377582 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:38.806581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:38.814437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1058/2150] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:38.820181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6fd37-e2f9-4b9e-a769-e8d2e3a3f157 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:38.843220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:38.844146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.844536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.844929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] PATH_INFO: `/v3/projects/03212885780843bf830a8f030756db97/users/d4d85df24c9e4312bbe2da197f55e116/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:38.868698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03212885780843bf830a8f030756db97, user_id=d4d85df24c9e4312bbe2da197f55e116, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:38.871889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:38.966955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1093/2151] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:38 2022] PUT /identity/v3/projects/03212885780843bf830a8f030756db97/users/d4d85df24c9e4312bbe2da197f55e116/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:38.982335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952ae1a-383d-4cb6-9788-d3df9e8c1c88 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:38.982335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952ae1a-383d-4cb6-9788-d3df9e8c1c88 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:38.982335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952ae1a-383d-4cb6-9788-d3df9e8c1c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:39.028983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d952ae1a-383d-4cb6-9788-d3df9e8c1c88 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:39.030364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d952ae1a-383d-4cb6-9788-d3df9e8c1c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:39.127385 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:39.162397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1059/2152] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:39.172923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:39.175087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:39.242653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e63e9-2363-4832-9199-4c6cbe9a9d70 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:39.246623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:39.247261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:39.247576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:39.247865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:39.261470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:39.262978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:39.278415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1094/2153] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:02:47.135996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619fc26-d66d-4b57-b9a7-3d2f0dc10a8c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:47.138248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619fc26-d66d-4b57-b9a7-3d2f0dc10a8c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:47.138248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619fc26-d66d-4b57-b9a7-3d2f0dc10a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:47.230282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1619fc26-d66d-4b57-b9a7-3d2f0dc10a8c None None] MFA Rules not processed for user `d4d85df24c9e4312bbe2da197f55e116`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:47.230282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1619fc26-d66d-4b57-b9a7-3d2f0dc10a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:47.385548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1060/2154] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:47.391327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:47.437093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e906cde-088f-4271-8915-df572b7a9428 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:47.455726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:47.456790 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:47.457687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:47.458204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:47.459268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:47.461529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:47.579594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1095/2155] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:02:47 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:02:47.583199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619fc26-d66d-4b57-b9a7-3d2f0dc10a8c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:47.599310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619fc26-d66d-4b57-b9a7-3d2f0dc10a8c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:47.715823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:47.716604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:47.716891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:47.717165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:47.717866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:47.719835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:47.896966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1061/2156] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:02:47 2022] POST /identity/v3/users => generated 295 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:02:47.900721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:47.918780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f1bc3-5543-4743-b13b-1c59a95c3b34 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:47.927328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:47.928972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:47.929569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:47.929943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:47.930781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:47.933231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:47.960605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1096/2157] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:47.965064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:47.980041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5658568-8245-401b-9766-ed0e4387eb8e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:47.992663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:47.996215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:47.996577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:47.996870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] PATH_INFO: `/v3/projects/5a25e70fd5074ac9981af72d76fd0662/users/ebd64c1ff3de4d298ba87f9298ad56fa/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:48.036766 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a25e70fd5074ac9981af72d76fd0662, user_id=ebd64c1ff3de4d298ba87f9298ad56fa, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:48.040598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:48.136390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1062/2158] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:47 2022] PUT /identity/v3/projects/5a25e70fd5074ac9981af72d76fd0662/users/ebd64c1ff3de4d298ba87f9298ad56fa/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:48.140393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:48.182179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7f58-7111-45d1-9174-65c8b2675f53 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:48.286528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337700d-9d2f-496e-bfdf-91a3381de732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:48.287330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337700d-9d2f-496e-bfdf-91a3381de732 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:48.287728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337700d-9d2f-496e-bfdf-91a3381de732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:48.287990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337700d-9d2f-496e-bfdf-91a3381de732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:48.288701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337700d-9d2f-496e-bfdf-91a3381de732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:48.290590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337700d-9d2f-496e-bfdf-91a3381de732 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:48.305341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1097/2159] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:02:48.309326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:48.329820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322113-201b-41f7-8747-193175ce4aec admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:48.336006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94471d6-9970-467c-8ae1-0b4dd81ba108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:48.336905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94471d6-9970-467c-8ae1-0b4dd81ba108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:48.337824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94471d6-9970-467c-8ae1-0b4dd81ba108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:48.337824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94471d6-9970-467c-8ae1-0b4dd81ba108 admin admin] PATH_INFO: `/v3/projects/5a25e70fd5074ac9981af72d76fd0662/users/ebd64c1ff3de4d298ba87f9298ad56fa/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:48.340310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94471d6-9970-467c-8ae1-0b4dd81ba108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a25e70fd5074ac9981af72d76fd0662, user_id=ebd64c1ff3de4d298ba87f9298ad56fa, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:48.343511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94471d6-9970-467c-8ae1-0b4dd81ba108 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:48.361858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1063/2160] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:02:48 2022] PUT /identity/v3/projects/5a25e70fd5074ac9981af72d76fd0662/users/ebd64c1ff3de4d298ba87f9298ad56fa/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:02:50.482357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:50.482357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:50.482357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:50.533015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] MFA Rules not processed for user `972bc4e9bc9e49b4a3bc2a693b05e4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:50.536301 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:50.571709 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:50.675383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1098/2161] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:56.736803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f42d-3f38-4450-8951-cd0bdf143801 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:56.737693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f42d-3f38-4450-8951-cd0bdf143801 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:56.737693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f42d-3f38-4450-8951-cd0bdf143801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:56.779080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0018f42d-3f38-4450-8951-cd0bdf143801 None None] MFA Rules not processed for user `ebd64c1ff3de4d298ba87f9298ad56fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:02:56.781160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0018f42d-3f38-4450-8951-cd0bdf143801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:56.791647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:02:57.034360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1064/2162] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:02:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:02:57.036350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:02:57.038808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:02:57.218197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531b1a8-1b5f-429a-95c3-5661301d1ac1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:02:57.232846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:02:57.234773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:02:57.235841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:02:57.236334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:02:57.371779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:02:57.375199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:02:57.390365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1099/2163] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:02:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:03:02.782037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d593eb-4eb2-44b0-ad46-de4826f37a81 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:02.782484 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d593eb-4eb2-44b0-ad46-de4826f37a81 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:02.782573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d593eb-4eb2-44b0-ad46-de4826f37a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:02.803117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-80d593eb-4eb2-44b0-ad46-de4826f37a81 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:02.804327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-80d593eb-4eb2-44b0-ad46-de4826f37a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:02.861086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1065/2164] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 08:03:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:03:02.874348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:02.875793 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:02.936471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:02.951132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0794ff4-387f-4825-8920-18d0e7962ea1 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:02.956907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c938d-d92b-48f6-970c-d4001fa67958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:02.957526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c938d-d92b-48f6-970c-d4001fa67958 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:02.958146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c938d-d92b-48f6-970c-d4001fa67958 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:02.958437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c938d-d92b-48f6-970c-d4001fa67958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:02.967310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c938d-d92b-48f6-970c-d4001fa67958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:02.969398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c938d-d92b-48f6-970c-d4001fa67958 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:02.978453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1100/2165] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:03:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:03:05.967858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d67e-fa60-4483-ae34-13fedbf53436 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:05.968354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d67e-fa60-4483-ae34-13fedbf53436 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:05.968571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9d67e-fa60-4483-ae34-13fedbf53436 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:05.969979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1066/2166] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:03:05.999796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:06.000361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:06.000536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:06.021000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] MFA Rules not processed for user `5b26619beabb4e8f96d9f08ecb107027`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:06.022539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:06.090674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1101/2167] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:03:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:03:06.102401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9d67e-fa60-4483-ae34-13fedbf53436 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:06.104353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c8c9d67e-fa60-4483-ae34-13fedbf53436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:06.124707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:06.126655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:06.131226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9d67e-fa60-4483-ae34-13fedbf53436 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:06.140580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:06.140580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:06.140580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:06.140580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:06.148159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:06.149804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:06.161960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1067/2168] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:03:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:03:06.221445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4491d8-6e85-45b0-ba27-127b971a1d7f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:06.227576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:06.227799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:06.228078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:06.228360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] PATH_INFO: `/v3/users/19c483a587bf423aaa0fdb9b4e24ce9d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:06.229707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c483a587bf423aaa0fdb9b4e24ce9d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:06.232332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:06.455658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c483a587bf423aaa0fdb9b4e24ce9d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:06.563214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c483a587bf423aaa0fdb9b4e24ce9d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:06.571558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c483a587bf423aaa0fdb9b4e24ce9d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:06.572199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c483a587bf423aaa0fdb9b4e24ce9d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:06.572911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a43f1-8863-46db-8810-2a6f785834a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c483a587bf423aaa0fdb9b4e24ce9d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:06.592307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1102/2169] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:03:06 2022] DELETE /identity/v3/users/19c483a587bf423aaa0fdb9b4e24ce9d => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:03:06.844878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:06.857833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175a324-42c6-4a17-94a5-93dcb78c5aa8 service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:06.922474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:06.923141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:06.923509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:06.923815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] PATH_INFO: `/v3/projects/c7089377e1ee4c64973fe9d8a1b59735` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:06.925001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7089377e1ee4c64973fe9d8a1b59735)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:06.927478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:07.036611 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:07.046760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7089377e1ee4c64973fe9d8a1b59735', 'request_id': 'req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:07.053691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7089377e1ee4c64973fe9d8a1b59735', 'request_id': 'req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:07.059765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7089377e1ee4c64973fe9d8a1b59735', 'request_id': 'req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:07.145561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] The token cache is being invalidate because project c7089377e1ee4c64973fe9d8a1b59735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:03:07.145847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:07.146483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81ca948c-1ad3-4e27-8ef5-636208a8e8ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:07.148120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1068/2170] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:03:06 2022] DELETE /identity/v3/projects/c7089377e1ee4c64973fe9d8a1b59735 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:03:07.153028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20cf88-5af9-4162-a6fb-6e276e555fab None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.153457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20cf88-5af9-4162-a6fb-6e276e555fab None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.153645 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20cf88-5af9-4162-a6fb-6e276e555fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.162361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:07.175897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6a20cf88-5af9-4162-a6fb-6e276e555fab None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:07.178035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6a20cf88-5af9-4162-a6fb-6e276e555fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:07.238098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1103/2171] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:07.246253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92e5c-1e4e-4326-a37d-a1e39528d8e7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.246543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92e5c-1e4e-4326-a37d-a1e39528d8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.246823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92e5c-1e4e-4326-a37d-a1e39528d8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.266759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7dd92e5c-1e4e-4326-a37d-a1e39528d8e7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:07.268050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7dd92e5c-1e4e-4326-a37d-a1e39528d8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:07.279153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1069/2172] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:07.283746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20cf88-5af9-4162-a6fb-6e276e555fab None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:07.294497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20cf88-5af9-4162-a6fb-6e276e555fab None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:07.299289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:07.299942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.300234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.300512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.301217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:03:07.302158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:07.304039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:07.316480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1104/2173] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:03:07.320986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92e5c-1e4e-4326-a37d-a1e39528d8e7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:07.333683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92e5c-1e4e-4326-a37d-a1e39528d8e7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:07.338707 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:07.339346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.339672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.339950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.340626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:07.342538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:07.442497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1070/2174] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:03:07 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:03:07.446464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:07.456606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fee91-de68-41b1-80fb-bdf760367907 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:07.518565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:07.519209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.519496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.519810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.521008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:07.523141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:07.592392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1105/2175] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:03:07 2022] POST /identity/v3/users => generated 285 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:03:07.596709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:07.608938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e86fe-164f-4cb8-aaf6-27a769a331fd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:07.616081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:07.616081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.616081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.616081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.616510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:07.617971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:07.629622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1071/2176] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:03:07.633524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:07.644058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5440-918c-4ae4-8a36-a5ca7206219e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:07.649939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651493e-771c-4991-8c14-b4006dc0de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:07.650557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651493e-771c-4991-8c14-b4006dc0de30 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.651011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651493e-771c-4991-8c14-b4006dc0de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.651442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651493e-771c-4991-8c14-b4006dc0de30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.652279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651493e-771c-4991-8c14-b4006dc0de30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:07.653989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651493e-771c-4991-8c14-b4006dc0de30 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:07.666266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1106/2177] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:03:07.670645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:07.682280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed642442-37c1-4b5d-a9ed-a4f96dd04b34 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:07.687170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:07.687932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.688226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.688509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] PATH_INFO: `/v3/projects/5c790f802069452f90843ae6a6da3896/users/9afc95c088114551bd369a399c13279a/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.704109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c790f802069452f90843ae6a6da3896, user_id=9afc95c088114551bd369a399c13279a, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:07.706816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:07.790964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1072/2178] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:03:07 2022] PUT /identity/v3/projects/5c790f802069452f90843ae6a6da3896/users/9afc95c088114551bd369a399c13279a/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:03:07.802201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.802507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.802856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.833889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] MFA Rules not processed for user `9afc95c088114551bd369a399c13279a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:07.835283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:07.897467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:07.902836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1107/2179] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:07.915718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:07.917525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:07.976237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af0cc-bf77-41c6-8905-4324f4202d2b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:07.979961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634636cc-3418-4683-a736-bf3f6c032794 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:07.980581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634636cc-3418-4683-a736-bf3f6c032794 service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:07.980868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634636cc-3418-4683-a736-bf3f6c032794 service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:07.981142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634636cc-3418-4683-a736-bf3f6c032794 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:07.989905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634636cc-3418-4683-a736-bf3f6c032794 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:07.991378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634636cc-3418-4683-a736-bf3f6c032794 service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:08.001153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1073/2180] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:03:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:03:17.107702 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:17.108723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:17.173333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8b745-a371-4b3d-8b8f-d92ebcc90926 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:17.177798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:17.179000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:17.179314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:17.179618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] PATH_INFO: `/v3/users/171e3adfcd8d40b9b593b7c6b426ad82` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:17.180823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] RBAC: Authorizing `identity:delete_user(user_id=171e3adfcd8d40b9b593b7c6b426ad82)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:17.182773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:17.383341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '171e3adfcd8d40b9b593b7c6b426ad82'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:17.475434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '171e3adfcd8d40b9b593b7c6b426ad82'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:17.483880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '171e3adfcd8d40b9b593b7c6b426ad82'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:17.484631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '171e3adfcd8d40b9b593b7c6b426ad82'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:17.485242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '171e3adfcd8d40b9b593b7c6b426ad82'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:17.503247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1108/2181] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:03:17 2022] DELETE /identity/v3/users/171e3adfcd8d40b9b593b7c6b426ad82 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:03:17.509053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:17.509596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:17.509596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:17.528565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:17.529773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:17.588588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1074/2182] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:17.598576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:17.600683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:17.657479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a48e3c-7c8a-4050-957d-452415f2957a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:17.661129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81163ee-46e3-486a-94a4-e3181feb5b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:17.661746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81163ee-46e3-486a-94a4-e3181feb5b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:17.662030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81163ee-46e3-486a-94a4-e3181feb5b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:17.662308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81163ee-46e3-486a-94a4-e3181feb5b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:17.672129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81163ee-46e3-486a-94a4-e3181feb5b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:17.673566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81163ee-46e3-486a-94a4-e3181feb5b22 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:17.683595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1109/2183] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:03:18.169392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:18.171130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:18.178271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:18.185120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bf17-17eb-4b4d-ad5e-ea031b57d555 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:18.190296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:18.191506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.192979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.193276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] PATH_INFO: `/v3/projects/b2a874e651ab429f9544d9247a0aa8aa` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.194546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a874e651ab429f9544d9247a0aa8aa)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:18.196526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:18.344618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a874e651ab429f9544d9247a0aa8aa', 'request_id': 'req-aabd9e45-a587-42ae-8839-a12c10dc52ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:18.351107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a874e651ab429f9544d9247a0aa8aa', 'request_id': 'req-aabd9e45-a587-42ae-8839-a12c10dc52ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:18.357364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a874e651ab429f9544d9247a0aa8aa', 'request_id': 'req-aabd9e45-a587-42ae-8839-a12c10dc52ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:18.435559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] The token cache is being invalidate because project b2a874e651ab429f9544d9247a0aa8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:03:18.436047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:18.436825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-aabd9e45-a587-42ae-8839-a12c10dc52ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:18.438760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1075/2184] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:03:18 2022] DELETE /identity/v3/projects/b2a874e651ab429f9544d9247a0aa8aa => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:03:18.444209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7014be0-ad05-4e6a-ac4a-d887623acbef None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.444723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7014be0-ad05-4e6a-ac4a-d887623acbef None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.444832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7014be0-ad05-4e6a-ac4a-d887623acbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.464711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d7014be0-ad05-4e6a-ac4a-d887623acbef None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:18.466421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d7014be0-ad05-4e6a-ac4a-d887623acbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:18.479428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:18.526982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1110/2185] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:18.535893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1619ab-7124-4ba2-ba0c-aedc5b1b1d18 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.536419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1619ab-7124-4ba2-ba0c-aedc5b1b1d18 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.536519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1619ab-7124-4ba2-ba0c-aedc5b1b1d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.558358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2f1619ab-7124-4ba2-ba0c-aedc5b1b1d18 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:18.559687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2f1619ab-7124-4ba2-ba0c-aedc5b1b1d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:18.567923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1076/2186] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:18.572415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7014be0-ad05-4e6a-ac4a-d887623acbef None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:18.582449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7014be0-ad05-4e6a-ac4a-d887623acbef None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:18.587430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:18.588075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.588364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.588643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.589340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:03:18.589652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:18.591265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:18.603402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1111/2187] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:03:18.607513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1619ab-7124-4ba2-ba0c-aedc5b1b1d18 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:18.617578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1619ab-7124-4ba2-ba0c-aedc5b1b1d18 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:18.622765 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:18.623437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.623781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.624089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.624829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:18.626864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:18.713798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1077/2188] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:03:18 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:03:18.717910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:18.728732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bacccc-d287-4d08-a340-761357be9e4a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:18.788181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:18.789747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.790141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.790425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.791125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:18.793098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:18.860148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1112/2189] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:03:18 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:03:18.864476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:18.875005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886fadc-00cf-4e58-8a0b-c4f04f2461fa admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:18.879721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:18.880335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.880623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.880899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.881648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:18.883463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:18.893462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1078/2190] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:03:18.898641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:18.909284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0dfa5-b6b0-45f2-b29a-eb71285e432e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:18.915008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c17466-7a52-4302-a2a8-95b81c4548f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:18.915650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c17466-7a52-4302-a2a8-95b81c4548f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.915947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c17466-7a52-4302-a2a8-95b81c4548f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.916230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c17466-7a52-4302-a2a8-95b81c4548f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.916928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c17466-7a52-4302-a2a8-95b81c4548f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:18.918544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c17466-7a52-4302-a2a8-95b81c4548f8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:18.929626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1113/2191] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:03:18.934002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:18.943962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7039e-a3c2-4e3b-a560-e15a38749850 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:18.948677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:18.949336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:18.949623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:18.949895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] PATH_INFO: `/v3/projects/86d7d8d9f331458cb5ddb5be26ece78c/users/08a36caefc5743229c793145e4b4e2a1/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:18.967499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d7d8d9f331458cb5ddb5be26ece78c, user_id=08a36caefc5743229c793145e4b4e2a1, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:18.970204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:19.037430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1079/2192] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:03:18 2022] PUT /identity/v3/projects/86d7d8d9f331458cb5ddb5be26ece78c/users/08a36caefc5743229c793145e4b4e2a1/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:03:19.049070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be58f5-5de7-4b6f-a3ec-4579ec543a5c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:19.049296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be58f5-5de7-4b6f-a3ec-4579ec543a5c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:19.049579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be58f5-5de7-4b6f-a3ec-4579ec543a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:19.079084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-70be58f5-5de7-4b6f-a3ec-4579ec543a5c None None] MFA Rules not processed for user `08a36caefc5743229c793145e4b4e2a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:19.080407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-70be58f5-5de7-4b6f-a3ec-4579ec543a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:19.146395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1114/2193] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:19.157791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:19.160798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:19.222287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb83f8-1614-43b8-9df2-40cea5b0985f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:19.226246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:19.226869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:19.227266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:19.227578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:19.236317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:19.237741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:19.247791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1080/2194] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:03:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:03:29.143384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:29.243798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:32.966293 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05925d5-e242-4be1-84e0-319f6597f893 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:32.966782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05925d5-e242-4be1-84e0-319f6597f893 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:32.966879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05925d5-e242-4be1-84e0-319f6597f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:32.986423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f05925d5-e242-4be1-84e0-319f6597f893 None None] MFA Rules not processed for user `d4d85df24c9e4312bbe2da197f55e116`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:32.987662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f05925d5-e242-4be1-84e0-319f6597f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:33.045436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1115/2195] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:35.552841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:35.554239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:35.624500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbec640-2640-4a76-a31a-35a709063ecf service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:35.629737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0919386-5b14-43e5-9fb2-a2e48f4eaf8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:35.630442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919386-5b14-43e5-9fb2-a2e48f4eaf8f service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:35.630803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919386-5b14-43e5-9fb2-a2e48f4eaf8f service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:35.631156 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919386-5b14-43e5-9fb2-a2e48f4eaf8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:35.633610 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b0919386-5b14-43e5-9fb2-a2e48f4eaf8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:35.640087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05925d5-e242-4be1-84e0-319f6597f893 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:35.646523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0919386-5b14-43e5-9fb2-a2e48f4eaf8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:35.648169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0919386-5b14-43e5-9fb2-a2e48f4eaf8f service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:35.659241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1081/2196] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:03:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:03:35.707255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05925d5-e242-4be1-84e0-319f6597f893 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:35.712683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:35.713332 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:35.713619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:35.713899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] PATH_INFO: `/v3/users/08a36caefc5743229c793145e4b4e2a1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:35.715081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a36caefc5743229c793145e4b4e2a1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:35.717064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:35.897756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a36caefc5743229c793145e4b4e2a1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:35.984761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a36caefc5743229c793145e4b4e2a1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:35.993196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a36caefc5743229c793145e4b4e2a1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:35.993859 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a36caefc5743229c793145e4b4e2a1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:35.994551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a36caefc5743229c793145e4b4e2a1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:36.013498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1116/2197] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:03:35 2022] DELETE /identity/v3/users/08a36caefc5743229c793145e4b4e2a1 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:03:36.017938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:36.018229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:36.018507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:36.037893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:36.039445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:36.100754 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1082/2198] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:03:36.110479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:36.111975 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:36.140478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:36.171354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9370-a38e-41c5-84a2-09981ebd4929 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:36.175317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d84194-f7fc-4af3-addb-18e35209289f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:36.177046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d84194-f7fc-4af3-addb-18e35209289f service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:36.177046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d84194-f7fc-4af3-addb-18e35209289f service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:36.177046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d84194-f7fc-4af3-addb-18e35209289f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:36.187119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d84194-f7fc-4af3-addb-18e35209289f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:36.188808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d84194-f7fc-4af3-addb-18e35209289f service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:36.200015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1117/2199] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:03:36.601588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:36.603335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:36.629954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ad58e-a9bd-4d26-8a3a-60538ed950ac None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:36.638261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:36.639063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:36.639459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:36.640341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] PATH_INFO: `/v3/projects/86d7d8d9f331458cb5ddb5be26ece78c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:36.641124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d7d8d9f331458cb5ddb5be26ece78c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:36.643180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:36.755700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d7d8d9f331458cb5ddb5be26ece78c', 'request_id': 'req-554febc4-3d81-4a8a-abb4-1914817f482f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:36.764997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d7d8d9f331458cb5ddb5be26ece78c', 'request_id': 'req-554febc4-3d81-4a8a-abb4-1914817f482f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:36.770844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d7d8d9f331458cb5ddb5be26ece78c', 'request_id': 'req-554febc4-3d81-4a8a-abb4-1914817f482f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:36.875486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] The token cache is being invalidate because project 86d7d8d9f331458cb5ddb5be26ece78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:03:36.875815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:36.876497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-554febc4-3d81-4a8a-abb4-1914817f482f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:03:36.878271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1083/2200] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:03:36 2022] DELETE /identity/v3/projects/86d7d8d9f331458cb5ddb5be26ece78c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:03:36.884074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23c8fc-ed12-48a2-8bf5-0df1078a0483 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:36.884634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23c8fc-ed12-48a2-8bf5-0df1078a0483 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:36.884880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23c8fc-ed12-48a2-8bf5-0df1078a0483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:36.906361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ab23c8fc-ed12-48a2-8bf5-0df1078a0483 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:36.907990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ab23c8fc-ed12-48a2-8bf5-0df1078a0483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:36.974533 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1118/2201] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:03:37.046876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab81a-62dc-4ed6-90db-7d37c15990ce None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.047172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab81a-62dc-4ed6-90db-7d37c15990ce None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.048202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab81a-62dc-4ed6-90db-7d37c15990ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.067969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-726ab81a-62dc-4ed6-90db-7d37c15990ce None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:37.069203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-726ab81a-62dc-4ed6-90db-7d37c15990ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:37.080685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1084/2202] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:37.084716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23c8fc-ed12-48a2-8bf5-0df1078a0483 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:37.095628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23c8fc-ed12-48a2-8bf5-0df1078a0483 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:37.102552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:37.102552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.102552 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.102934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.103320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:03:37.103661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:37.105310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:37.117625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1119/2203] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:03:37.122206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ab81a-62dc-4ed6-90db-7d37c15990ce None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:37.132224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ab81a-62dc-4ed6-90db-7d37c15990ce None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:37.137979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:37.138702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.139001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.139261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.139984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:37.142003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:37.146772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:37.245685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1085/2204] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:03:37 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:03:37.249750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:37.261767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9ed9d-98df-4468-81f0-aa8a2bc2db17 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:37.319780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:37.321050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.321563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.321935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.322603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:37.324613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:37.403090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1120/2205] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:03:37 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:03:37.407302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:37.418748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf01f9-a630-4f5c-8718-e9e0db1255f0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:37.423862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:37.424499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.425182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.425471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.426622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:37.428284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:37.445258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:37.451350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1086/2206] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:03:37.462889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169953-8f82-490f-abc7-4800e202d760 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:37.473159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5260d9-3bfe-4d0f-ae61-b2ff7dd86881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:37.474442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5260d9-3bfe-4d0f-ae61-b2ff7dd86881 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.474442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5260d9-3bfe-4d0f-ae61-b2ff7dd86881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.474442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5260d9-3bfe-4d0f-ae61-b2ff7dd86881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.474999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5260d9-3bfe-4d0f-ae61-b2ff7dd86881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:37.476706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5260d9-3bfe-4d0f-ae61-b2ff7dd86881 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:37.494347 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:37.495430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1121/2207] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:03:37.505099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0362f-75e6-45b0-963b-0157b31b70f8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:37.510650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:37.511346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.511676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.511969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] PATH_INFO: `/v3/projects/02bd4344d2ce4ea19f3bdec40d9619c3/users/abdb605f4270460cb54d7f6ad079ec96/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.529929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=02bd4344d2ce4ea19f3bdec40d9619c3, user_id=abdb605f4270460cb54d7f6ad079ec96, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:37.532761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:37.607736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1087/2208] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:03:37 2022] PUT /identity/v3/projects/02bd4344d2ce4ea19f3bdec40d9619c3/users/abdb605f4270460cb54d7f6ad079ec96/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:03:37.618775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.619069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.619347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.653253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] MFA Rules not processed for user `abdb605f4270460cb54d7f6ad079ec96`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:03:37.654586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:37.677497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:37.723260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1122/2209] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:03:37.735441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:03:37.737171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:03:37.815369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33788-375f-4ed1-ae4f-929451c55a4c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:03:37.819281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:03:37.819958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:03:37.820248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:03:37.821135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:03:37.830759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:03:37.832319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:03:37.844773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1088/2210] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:03:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:03:47.839629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:03:47.865010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=17 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:07.102090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:07.103495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:07.161675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92117b53-8b93-4df1-8165-e410e028c246 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:07.165657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cc1f4-1cf1-4ee4-8d05-a253ca71b914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:07.166309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cc1f4-1cf1-4ee4-8d05-a253ca71b914 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:07.166623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cc1f4-1cf1-4ee4-8d05-a253ca71b914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:07.166899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cc1f4-1cf1-4ee4-8d05-a253ca71b914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:07.168710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2d4cc1f4-1cf1-4ee4-8d05-a253ca71b914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:07.231356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cc1f4-1cf1-4ee4-8d05-a253ca71b914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:07.232888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cc1f4-1cf1-4ee4-8d05-a253ca71b914 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:07.242334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1123/2211] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:04:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:04:14.240396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:14.302818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bf1fb-2d4f-4297-a802-c6b9449a3b9e service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:14.307344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:14.308006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:14.308292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:14.308568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] PATH_INFO: `/v3/users/abdb605f4270460cb54d7f6ad079ec96` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:14.309721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdb605f4270460cb54d7f6ad079ec96)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:14.311810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:14.478595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdb605f4270460cb54d7f6ad079ec96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:14.485870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdb605f4270460cb54d7f6ad079ec96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:14.566846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdb605f4270460cb54d7f6ad079ec96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:14.567562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdb605f4270460cb54d7f6ad079ec96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:14.580751 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdb605f4270460cb54d7f6ad079ec96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:14.586725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1089/2212] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:04:14 2022] DELETE /identity/v3/users/abdb605f4270460cb54d7f6ad079ec96 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:14.591223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:14.592409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:14.592510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:14.612922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:14.614149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:14.674156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1124/2213] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:14.683793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:14.685212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:14.755243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1efbb-613d-4905-a322-fac41c2c680d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:14.755243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395fba6-1a34-4464-b54a-52ca21310433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:14.755243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fba6-1a34-4464-b54a-52ca21310433 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:14.755243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fba6-1a34-4464-b54a-52ca21310433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:14.755651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fba6-1a34-4464-b54a-52ca21310433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:14.765856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395fba6-1a34-4464-b54a-52ca21310433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:14.767418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395fba6-1a34-4464-b54a-52ca21310433 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:14.777624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1090/2214] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:04:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:04:15.228350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:15.229791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:15.249980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440400b-5020-4f28-92eb-51724f17f288 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:15.249980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:15.249980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:15.249980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:15.250358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] PATH_INFO: `/v3/projects/02bd4344d2ce4ea19f3bdec40d9619c3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:15.251247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02bd4344d2ce4ea19f3bdec40d9619c3)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:15.253324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:15.375668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:15.378511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02bd4344d2ce4ea19f3bdec40d9619c3', 'request_id': 'req-0cc724ee-c2aa-4907-b760-b8740c0dfa77'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:15.449451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02bd4344d2ce4ea19f3bdec40d9619c3', 'request_id': 'req-0cc724ee-c2aa-4907-b760-b8740c0dfa77'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:15.457612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02bd4344d2ce4ea19f3bdec40d9619c3', 'request_id': 'req-0cc724ee-c2aa-4907-b760-b8740c0dfa77'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:15.473524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] The token cache is being invalidate because project 02bd4344d2ce4ea19f3bdec40d9619c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:04:15.473823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:15.474477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:15.476133 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1125/2215] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:04:15 2022] DELETE /identity/v3/projects/02bd4344d2ce4ea19f3bdec40d9619c3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:24.773102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:25.470544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:25.566236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b83bc-ec12-458b-a4f0-0f9afa27e829 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:25.566528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b83bc-ec12-458b-a4f0-0f9afa27e829 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:25.566807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b83bc-ec12-458b-a4f0-0f9afa27e829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:25.586933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-217b83bc-ec12-458b-a4f0-0f9afa27e829 None None] MFA Rules not processed for user `0d2c7d95047445ee9d44640b5372e0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:25.588142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-217b83bc-ec12-458b-a4f0-0f9afa27e829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:25.646858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1091/2216] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:28.672633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:28.674590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:28.745353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc724ee-c2aa-4907-b760-b8740c0dfa77 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:28.761776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:28.762580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:28.762580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:28.762960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] PATH_INFO: `/v3/users/9afc95c088114551bd369a399c13279a` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:28.767462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afc95c088114551bd369a399c13279a)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:28.769525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:28.922523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afc95c088114551bd369a399c13279a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:28.984846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afc95c088114551bd369a399c13279a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:28.993350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afc95c088114551bd369a399c13279a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:28.994031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afc95c088114551bd369a399c13279a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:28.994659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afc95c088114551bd369a399c13279a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:29.013347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1126/2217] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:04:28 2022] DELETE /identity/v3/users/9afc95c088114551bd369a399c13279a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:04:29.017869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:29.018353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:29.018517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:29.038311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:29.039947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:29.102502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1092/2218] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:29.114187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:29.114187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:29.172535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6ed40-396e-41f9-ab47-b8b6b568fb73 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:29.176426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581ab39-d3de-4ee6-8a06-e00017822795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:29.177044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581ab39-d3de-4ee6-8a06-e00017822795 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:29.177331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581ab39-d3de-4ee6-8a06-e00017822795 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:29.177609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581ab39-d3de-4ee6-8a06-e00017822795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:29.186849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581ab39-d3de-4ee6-8a06-e00017822795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:29.188383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581ab39-d3de-4ee6-8a06-e00017822795 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:29.199482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1127/2219] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:04:29.643115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:29.649237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:29.664198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb88bb6-899f-4237-b3af-847d94726c83 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:29.670729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:29.672815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:29.684815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:29.685121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] PATH_INFO: `/v3/projects/5c790f802069452f90843ae6a6da3896` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:29.690378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c790f802069452f90843ae6a6da3896)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:29.692380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:29.820640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c790f802069452f90843ae6a6da3896', 'request_id': 'req-5db77469-ef90-468a-8b60-404b858de3ba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:29.827213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c790f802069452f90843ae6a6da3896', 'request_id': 'req-5db77469-ef90-468a-8b60-404b858de3ba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:29.833623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c790f802069452f90843ae6a6da3896', 'request_id': 'req-5db77469-ef90-468a-8b60-404b858de3ba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:29.899234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] The token cache is being invalidate because project 5c790f802069452f90843ae6a6da3896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:04:29.899666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:29.900366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5db77469-ef90-468a-8b60-404b858de3ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:29.901905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1093/2220] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:04:29 2022] DELETE /identity/v3/projects/5c790f802069452f90843ae6a6da3896 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:29.906983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec2601-8058-4dcb-9e4f-9400f43208ce None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:29.908185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec2601-8058-4dcb-9e4f-9400f43208ce None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:29.908185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec2601-8058-4dcb-9e4f-9400f43208ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:29.928573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1dec2601-8058-4dcb-9e4f-9400f43208ce None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:29.929787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1dec2601-8058-4dcb-9e4f-9400f43208ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:29.948683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:29.996239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1128/2221] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:30.005397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee113e4-5174-4852-b92d-2bcd744f1baa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.005776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee113e4-5174-4852-b92d-2bcd744f1baa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.006126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee113e4-5174-4852-b92d-2bcd744f1baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.027401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4ee113e4-5174-4852-b92d-2bcd744f1baa None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:30.028649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4ee113e4-5174-4852-b92d-2bcd744f1baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:30.037401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1094/2222] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:30.042177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec2601-8058-4dcb-9e4f-9400f43208ce None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:30.053149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec2601-8058-4dcb-9e4f-9400f43208ce None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:30.057881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:30.058504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.058792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.059074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.059844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:04:30.060155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:30.061782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:30.077074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1129/2223] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:04:30.078154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee113e4-5174-4852-b92d-2bcd744f1baa None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:30.087655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee113e4-5174-4852-b92d-2bcd744f1baa None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:30.092395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:30.093013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.093303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.093580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.094250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:30.096760 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:30.210988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:30.217467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1095/2224] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:04:30 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:04:30.222577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:30.232926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df9fd-4228-4803-a538-0b300175f473 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:30.293304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:30.294265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.294706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.295132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.296034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:30.298175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:30.392212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1130/2225] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:04:30 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:04:30.396532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:30.408257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff2151-e034-44af-bbf5-c7a31cb5f016 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:30.413304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:30.413983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.414266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.416705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.416705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:30.417071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:30.431264 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1096/2226] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:04:30.432580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:30.442735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a845cf7-f9d2-410f-a142-f494eb1e4240 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:30.447517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b23f96-1700-4a2e-8c5b-cfdbb7d66269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:30.448175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23f96-1700-4a2e-8c5b-cfdbb7d66269 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.448462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23f96-1700-4a2e-8c5b-cfdbb7d66269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.448739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23f96-1700-4a2e-8c5b-cfdbb7d66269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.449445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b23f96-1700-4a2e-8c5b-cfdbb7d66269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:30.451052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b23f96-1700-4a2e-8c5b-cfdbb7d66269 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:30.462764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1131/2227] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:04:30.467071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:30.477857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675de4f8-fdee-4867-89fe-8b7d7889dc98 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:30.482445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:30.483097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.483378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.483681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] PATH_INFO: `/v3/projects/f40f7d8cdf544abc94b52eaf668d1243/users/495b65d46d964135a36f506b7be98f67/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.498699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40f7d8cdf544abc94b52eaf668d1243, user_id=495b65d46d964135a36f506b7be98f67, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:30.501940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:30.581857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1097/2228] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:04:30 2022] PUT /identity/v3/projects/f40f7d8cdf544abc94b52eaf668d1243/users/495b65d46d964135a36f506b7be98f67/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:04:30.592278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397e361-9655-45f9-8a54-30444a831cc8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.592571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397e361-9655-45f9-8a54-30444a831cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.592879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397e361-9655-45f9-8a54-30444a831cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.620654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c397e361-9655-45f9-8a54-30444a831cc8 None None] MFA Rules not processed for user `495b65d46d964135a36f506b7be98f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:30.621875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c397e361-9655-45f9-8a54-30444a831cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:30.698932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1132/2229] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:30.721580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:30.723418 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:30.794696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e80a5-14f1-4ac4-ba8b-cc7807436577 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:30.798579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35209fc9-840f-431f-b24d-64be77060d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:30.799216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35209fc9-840f-431f-b24d-64be77060d6a service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:30.799503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35209fc9-840f-431f-b24d-64be77060d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:30.799813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35209fc9-840f-431f-b24d-64be77060d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:30.809569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35209fc9-840f-431f-b24d-64be77060d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:30.810981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35209fc9-840f-431f-b24d-64be77060d6a service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:30.823682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1098/2230] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:04:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:04:34.959681 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397e361-9655-45f9-8a54-30444a831cc8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:34.978794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:35.024255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397e361-9655-45f9-8a54-30444a831cc8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:35.029064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:35.030280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:35.030280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:35.030280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] PATH_INFO: `/v3/users/495b65d46d964135a36f506b7be98f67` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:35.031394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=495b65d46d964135a36f506b7be98f67)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:35.033906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:35.237203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495b65d46d964135a36f506b7be98f67'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:35.307231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495b65d46d964135a36f506b7be98f67'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:35.315069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495b65d46d964135a36f506b7be98f67'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:35.315770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495b65d46d964135a36f506b7be98f67'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:35.316401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495b65d46d964135a36f506b7be98f67'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:35.336591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1133/2231] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:04:34 2022] DELETE /identity/v3/users/495b65d46d964135a36f506b7be98f67 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:35.341167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:35.341666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:35.341767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:35.362386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:35.363645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:35.423623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1099/2232] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:35.433151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:35.434604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:35.492300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0207c8-8006-4bbd-826a-489cbb792024 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:35.496060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b3cc2-2df8-4032-b3df-b9d024ff675c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:35.496711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b3cc2-2df8-4032-b3df-b9d024ff675c service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:35.497000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b3cc2-2df8-4032-b3df-b9d024ff675c service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:35.497281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b3cc2-2df8-4032-b3df-b9d024ff675c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:35.506642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b3cc2-2df8-4032-b3df-b9d024ff675c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:35.508127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b3cc2-2df8-4032-b3df-b9d024ff675c service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:35.518969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1134/2233] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:04:36.020598 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:36.022029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:36.035748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247067c1-6e0a-46a6-8dd6-4f03fe71845e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:36.040676 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:36.041318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.041607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.041877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] PATH_INFO: `/v3/projects/f40f7d8cdf544abc94b52eaf668d1243` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.043043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40f7d8cdf544abc94b52eaf668d1243)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:36.045809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:36.147752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40f7d8cdf544abc94b52eaf668d1243', 'request_id': 'req-e06ca592-d597-4106-b149-743842368f87'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:36.154112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40f7d8cdf544abc94b52eaf668d1243', 'request_id': 'req-e06ca592-d597-4106-b149-743842368f87'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:36.160341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40f7d8cdf544abc94b52eaf668d1243', 'request_id': 'req-e06ca592-d597-4106-b149-743842368f87'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:36.162031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:36.226564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] The token cache is being invalidate because project f40f7d8cdf544abc94b52eaf668d1243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:04:36.226856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:36.227483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e06ca592-d597-4106-b149-743842368f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:36.229075 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1100/2234] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:04:36 2022] DELETE /identity/v3/projects/f40f7d8cdf544abc94b52eaf668d1243 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:36.236489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ab772-6909-4432-8de1-0268b61dea24 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.236489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ab772-6909-4432-8de1-0268b61dea24 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.236489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ab772-6909-4432-8de1-0268b61dea24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.256523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-569ab772-6909-4432-8de1-0268b61dea24 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:36.257741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-569ab772-6909-4432-8de1-0268b61dea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:36.293670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:36.319830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1135/2235] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:36.327857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3f97-c248-47e1-9081-4ade3ed2355a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.328154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3f97-c248-47e1-9081-4ade3ed2355a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.328466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3f97-c248-47e1-9081-4ade3ed2355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.349232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-094a3f97-c248-47e1-9081-4ade3ed2355a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:36.350459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-094a3f97-c248-47e1-9081-4ade3ed2355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:36.358709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1101/2236] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:04:36.363705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ab772-6909-4432-8de1-0268b61dea24 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:36.373951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ab772-6909-4432-8de1-0268b61dea24 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:36.379326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:36.379982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.380262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.380648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.381385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:04:36.381803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:36.383415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:36.397554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1136/2237] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:04:36.402112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3f97-c248-47e1-9081-4ade3ed2355a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:36.412858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3f97-c248-47e1-9081-4ade3ed2355a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:36.420233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:36.420233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.420233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.420233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.420596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:36.421715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:36.554973 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1102/2238] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:04:36 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:04:36.559285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:36.569912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c689f39-1ef7-4da0-9995-e3ce87831075 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:36.626642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:36.627299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.627682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.627965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.628630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:36.630565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:36.707735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1137/2239] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:04:36 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:04:36.712139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:36.723550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78513c28-4615-43e3-b2c6-43575bb8e9b8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:36.728341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:36.728965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.729364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.729648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.730352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:36.732016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:36.742613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1103/2240] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:04:36.746804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:36.757263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c99b1-8d72-4a1f-871e-017c9b380f5a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:36.763341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9353-fbfe-4f4c-8e54-fa5901ffd636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:36.763501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9353-fbfe-4f4c-8e54-fa5901ffd636 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.763821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9353-fbfe-4f4c-8e54-fa5901ffd636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.764099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9353-fbfe-4f4c-8e54-fa5901ffd636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.764796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9353-fbfe-4f4c-8e54-fa5901ffd636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:36.766485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9353-fbfe-4f4c-8e54-fa5901ffd636 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:36.777766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1138/2241] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:04:36.782166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:36.793756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ad2f5-5c9d-41c9-80cd-abe5c16a5c81 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:36.799407 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:36.800117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:36.800505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:36.800787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] PATH_INFO: `/v3/projects/4514412925584d0b9046d23576eaeab1/users/4861dabf60fc4399ae33ca6b62052ff6/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:36.816665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4514412925584d0b9046d23576eaeab1, user_id=4861dabf60fc4399ae33ca6b62052ff6, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:36.819358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:36.934820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1104/2242] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:04:36 2022] PUT /identity/v3/projects/4514412925584d0b9046d23576eaeab1/users/4861dabf60fc4399ae33ca6b62052ff6/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:37.007608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce71cb1-75c0-420d-9b09-149be68d5340 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:37.007950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce71cb1-75c0-420d-9b09-149be68d5340 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:37.008232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce71cb1-75c0-420d-9b09-149be68d5340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:37.050297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5ce71cb1-75c0-420d-9b09-149be68d5340 None None] MFA Rules not processed for user `4861dabf60fc4399ae33ca6b62052ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:37.051550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5ce71cb1-75c0-420d-9b09-149be68d5340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:37.118478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1139/2243] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:37.130468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:37.132267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:37.192305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074c3e-b2cc-4227-9a86-5c8e62d8db66 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:37.196061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3592eca-a878-4742-b637-09d6670e0ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:37.196719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3592eca-a878-4742-b637-09d6670e0ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:37.197007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3592eca-a878-4742-b637-09d6670e0ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:37.197287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3592eca-a878-4742-b637-09d6670e0ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:37.206135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3592eca-a878-4742-b637-09d6670e0ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:37.207615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3592eca-a878-4742-b637-09d6670e0ae5 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:37.216479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1105/2244] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:04:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:04:37.242089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce71cb1-75c0-420d-9b09-149be68d5340 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:37.281788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:37.305415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce71cb1-75c0-420d-9b09-149be68d5340 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:37.310203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:37.310843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:37.311126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:37.311401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] PATH_INFO: `/v3/users/4861dabf60fc4399ae33ca6b62052ff6` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:37.312597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4861dabf60fc4399ae33ca6b62052ff6)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:37.315177 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:37.524252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4861dabf60fc4399ae33ca6b62052ff6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:37.606867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4861dabf60fc4399ae33ca6b62052ff6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:37.614410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4861dabf60fc4399ae33ca6b62052ff6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:37.615052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4861dabf60fc4399ae33ca6b62052ff6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:37.615805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4861dabf60fc4399ae33ca6b62052ff6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:37.633407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1140/2245] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:04:37 2022] DELETE /identity/v3/users/4861dabf60fc4399ae33ca6b62052ff6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:37.638016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:37.638505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:37.638623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:37.657920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:04:37.659134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:37.717811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1106/2246] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:04:37.727749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:37.729183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:37.787376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca43b3-4985-400e-90bd-5b4e5c6dcb82 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:37.791219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:37.791867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:37.792153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:37.792458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:37.801899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:37.803356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:37.813387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1141/2247] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:04:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:04:38.305334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:04:38.306342 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:04:38.328367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e632ce0-41b6-4291-9b27-3a2d9b8b07ee None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:04:38.334023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:04:38.334323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:04:38.334608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:04:38.334887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] PATH_INFO: `/v3/projects/4514412925584d0b9046d23576eaeab1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:04:38.336119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4514412925584d0b9046d23576eaeab1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:04:38.338080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:04:38.451542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:38.471063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4514412925584d0b9046d23576eaeab1', 'request_id': 'req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:38.477596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4514412925584d0b9046d23576eaeab1', 'request_id': 'req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:38.483172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4514412925584d0b9046d23576eaeab1', 'request_id': 'req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:38.547644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] The token cache is being invalidate because project 4514412925584d0b9046d23576eaeab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:04:38.547947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:38.549246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:04:38.551060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1107/2248] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:04:38 2022] DELETE /identity/v3/projects/4514412925584d0b9046d23576eaeab1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:04:47.808635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:04:48.622416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:05:28.505135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:05:28.506644 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:28.576228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5dfec-ec31-46cc-9f7f-016174ddc0db service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:05:28.632560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:05:28.633191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:05:28.633476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:05:28.633799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:05:28.636116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:28.706335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:05:28.707938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:05:28.718914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1142/2249] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:05:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:05:28.960874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:05:28.962466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:29.020569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0dc3a-57bb-45b3-8003-5233c8e96a3d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:05:29.024313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:05:29.025013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:05:29.025313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:05:29.025572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:05:29.027372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:29.041894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:05:29.043324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:05:29.055004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1108/2250] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:05:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:05:33.252489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:05:33.253914 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:33.269879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bb7a0-0124-4a49-9db9-faf5bafba59e service glance] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:05:33.275346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c11ca5-3717-4a61-a847-c248c02903b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:05:33.276002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c11ca5-3717-4a61-a847-c248c02903b7 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:05:33.276267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c11ca5-3717-4a61-a847-c248c02903b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:05:33.276540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c11ca5-3717-4a61-a847-c248c02903b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:05:33.278257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-89c11ca5-3717-4a61-a847-c248c02903b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:33.333557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:05:33.335795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c11ca5-3717-4a61-a847-c248c02903b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:05:33.337413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c11ca5-3717-4a61-a847-c248c02903b7 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:05:33.346320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1143/2251] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:05:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:05:39.049480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=26 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:05:43.157297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:05:43.167067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f61df-12c7-4638-a14d-c5103fa21605 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:05:43.171037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:05:43.171715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:05:43.172009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:05:43.172373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:05:43.174324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:43.185232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:05:43.186684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:05:43.202721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1109/2252] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:05:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:05:43.362475 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:05:53.198712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:05:54.181314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99d1d8-0827-4425-a556-65b584ae0605 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:05:54.186383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99d1d8-0827-4425-a556-65b584ae0605 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:05:54.186383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99d1d8-0827-4425-a556-65b584ae0605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:05:54.202419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6e99d1d8-0827-4425-a556-65b584ae0605 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:05:54.203656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6e99d1d8-0827-4425-a556-65b584ae0605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:54.258951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1144/2253] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:05:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:05:54.407459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:05:54.408909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:54.479918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0b3c7-7145-468d-b51d-2222da510305 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:05:54.484938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:05:54.485560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:05:54.485846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:05:54.486122 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:05:54.494958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:05:54.496578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:05:54.504564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1110/2254] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:05:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:05:59.704374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74071723-4898-493f-8cac-ee8c8dd2186b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:05:59.704908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74071723-4898-493f-8cac-ee8c8dd2186b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:05:59.704996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74071723-4898-493f-8cac-ee8c8dd2186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:05:59.728569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-74071723-4898-493f-8cac-ee8c8dd2186b None None] MFA Rules not processed for user `972bc4e9bc9e49b4a3bc2a693b05e4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:05:59.729730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-74071723-4898-493f-8cac-ee8c8dd2186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:05:59.737535 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1145/2255] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:05:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:06:04.501409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=32 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:06:06.710835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:06.713933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:06.731234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82d43-0b19-4b08-80f0-46a89d6536f5 service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:06.740356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:06.740356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:06.740356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:06.740356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:06.746953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:06.757972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:06.758310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:06.770067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1111/2256] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:09.720986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=28 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:06:10.856134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74071723-4898-493f-8cac-ee8c8dd2186b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:10.870888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74071723-4898-493f-8cac-ee8c8dd2186b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:10.876960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c433d-16b0-4b1f-bebd-2997b5c64174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:10.877607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c433d-16b0-4b1f-bebd-2997b5c64174 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:10.877873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c433d-16b0-4b1f-bebd-2997b5c64174 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:10.878167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c433d-16b0-4b1f-bebd-2997b5c64174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:10.880656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ef0c433d-16b0-4b1f-bebd-2997b5c64174 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:10.893283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c433d-16b0-4b1f-bebd-2997b5c64174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:10.894918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c433d-16b0-4b1f-bebd-2997b5c64174 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:10.906020 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1146/2257] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:11.906251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:11.915772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c1d82-26d3-486b-bae2-ab9749496f63 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:11.919681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee75524-a3dd-4306-9481-4ef34bf9aacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:11.920293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75524-a3dd-4306-9481-4ef34bf9aacb service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:11.920580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75524-a3dd-4306-9481-4ef34bf9aacb service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:11.920887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75524-a3dd-4306-9481-4ef34bf9aacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:11.922629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4ee75524-a3dd-4306-9481-4ef34bf9aacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:11.932067 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee75524-a3dd-4306-9481-4ef34bf9aacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:11.933501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee75524-a3dd-4306-9481-4ef34bf9aacb service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:11.941368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1112/2258] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:12.363553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e27bd-035a-4012-ba1c-a82f8e9ac7e6 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.364107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e27bd-035a-4012-ba1c-a82f8e9ac7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.364418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e27bd-035a-4012-ba1c-a82f8e9ac7e6 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.365887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1147/2259] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:06:12.372550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011712e-0518-4315-ab18-13b93f2555ff None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.372839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011712e-0518-4315-ab18-13b93f2555ff None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.373116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011712e-0518-4315-ab18-13b93f2555ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.393427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2011712e-0518-4315-ab18-13b93f2555ff None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:06:12.394639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2011712e-0518-4315-ab18-13b93f2555ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:12.404424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1113/2260] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:06:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:06:12.416187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e27bd-035a-4012-ba1c-a82f8e9ac7e6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:12.418152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aa4e27bd-035a-4012-ba1c-a82f8e9ac7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:12.431928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e27bd-035a-4012-ba1c-a82f8e9ac7e6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:12.435410 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadb6d1-6d31-4a56-b822-8c730b45ed74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:12.436046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb6d1-6d31-4a56-b822-8c730b45ed74 service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.436330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb6d1-6d31-4a56-b822-8c730b45ed74 service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.436605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb6d1-6d31-4a56-b822-8c730b45ed74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.446854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcadb6d1-6d31-4a56-b822-8c730b45ed74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:12.448320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcadb6d1-6d31-4a56-b822-8c730b45ed74 service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:12.459040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1148/2261] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:12.648258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296ca3-a759-4ee5-8c5b-e63e42cb0357 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.648553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296ca3-a759-4ee5-8c5b-e63e42cb0357 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.648835 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296ca3-a759-4ee5-8c5b-e63e42cb0357 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.650230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1114/2262] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:06:12.654801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1bdcc-db93-4dc2-87a9-15c416f153a9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.655097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1bdcc-db93-4dc2-87a9-15c416f153a9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.655377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1bdcc-db93-4dc2-87a9-15c416f153a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.675866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c0a1bdcc-db93-4dc2-87a9-15c416f153a9 None None] MFA Rules not processed for user `ad3cf8a3e0a346fa867445d6b6300447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:06:12.677068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c0a1bdcc-db93-4dc2-87a9-15c416f153a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:12.684305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1149/2263] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:06:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:06:12.696723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc296ca3-a759-4ee5-8c5b-e63e42cb0357 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:12.707678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc296ca3-a759-4ee5-8c5b-e63e42cb0357 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:12.711887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac424d-8067-4357-b6a0-1934707b2ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:12.712434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac424d-8067-4357-b6a0-1934707b2ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.712722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac424d-8067-4357-b6a0-1934707b2ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.713001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac424d-8067-4357-b6a0-1934707b2ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.720085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac424d-8067-4357-b6a0-1934707b2ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:12.721519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac424d-8067-4357-b6a0-1934707b2ee8 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:12.729886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1115/2264] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:12.813212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb3b69-2b6a-46fa-84d8-4d754afc2fd8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.813582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb3b69-2b6a-46fa-84d8-4d754afc2fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.813922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb3b69-2b6a-46fa-84d8-4d754afc2fd8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.815378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1150/2265] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:06:12.820653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.820950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.821245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.841174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:06:12.842379 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:12.852384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1116/2266] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:06:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:06:12.862499 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb3b69-2b6a-46fa-84d8-4d754afc2fd8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:12.873715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb3b69-2b6a-46fa-84d8-4d754afc2fd8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:12.877367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a781be-a628-4f50-9806-ba4db4b4e1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:12.877973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a781be-a628-4f50-9806-ba4db4b4e1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:12.878260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a781be-a628-4f50-9806-ba4db4b4e1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:12.878539 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a781be-a628-4f50-9806-ba4db4b4e1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:12.889159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a781be-a628-4f50-9806-ba4db4b4e1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:12.890599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a781be-a628-4f50-9806-ba4db4b4e1a0 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:12.901950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1151/2267] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:12.964021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:12.965345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:13.033983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdec374-39b1-4db9-abb2-0157cf3eb7ad None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:13.039809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:13.040429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:13.040718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:13.040999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:13.043021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:13.054468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:13.056099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:13.065826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1117/2268] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:16.698747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524051-e9bb-4a7e-98b6-7a0072ed7822 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:16.699222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524051-e9bb-4a7e-98b6-7a0072ed7822 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:16.699324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524051-e9bb-4a7e-98b6-7a0072ed7822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:16.718775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5b524051-e9bb-4a7e-98b6-7a0072ed7822 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:06:16.720046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5b524051-e9bb-4a7e-98b6-7a0072ed7822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:16.727887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1152/2269] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:06:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:06:16.739622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:16.750656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd142d-7dbe-4be5-8ddd-9d673c2046ea service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:16.755672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:16.756299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:16.756734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:16.757041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:16.765649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:16.767247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:16.775756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1118/2270] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:06:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:06:23.892684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524051-e9bb-4a7e-98b6-7a0072ed7822 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:23.906621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524051-e9bb-4a7e-98b6-7a0072ed7822 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:23.913333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d4424-fc22-48a0-9d86-ead668c606b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:23.914056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d4424-fc22-48a0-9d86-ead668c606b9 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:23.914463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d4424-fc22-48a0-9d86-ead668c606b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:23.914811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d4424-fc22-48a0-9d86-ead668c606b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:23.916853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1c9d4424-fc22-48a0-9d86-ead668c606b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:23.980979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d4424-fc22-48a0-9d86-ead668c606b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:23.982742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d4424-fc22-48a0-9d86-ead668c606b9 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:23.992800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1153/2271] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:26.771955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=62 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:06:33.988155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=52 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:06:38.765888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:38.777904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19dfcc-161e-467a-a978-fd3e6cb95489 service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:38.783417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:38.784057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:38.784345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:38.784625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:38.786371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:38.846655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:38.848294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:38.861499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1119/2272] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:46.426789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37b982-f5e0-4bdf-9268-98849d584748 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:46.427283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37b982-f5e0-4bdf-9268-98849d584748 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:46.427390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37b982-f5e0-4bdf-9268-98849d584748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:46.447129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-aa37b982-f5e0-4bdf-9268-98849d584748 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:06:46.448368 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aa37b982-f5e0-4bdf-9268-98849d584748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:46.455471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1154/2273] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 08:06:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:06:46.468645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:46.480578 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2061abb-41c1-48d1-95cb-9b8fba5cde70 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:46.485643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:46.486254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:46.486544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:46.486846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:46.495229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:46.496964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:46.505983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1120/2274] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:06:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 08:06:55.492679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb7f6-efa0-4b76-9337-ec0ecd685bd5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:55.493217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb7f6-efa0-4b76-9337-ec0ecd685bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:55.493217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb7f6-efa0-4b76-9337-ec0ecd685bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:55.513146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-868fb7f6-efa0-4b76-9337-ec0ecd685bd5 None None] MFA Rules not processed for user `0d2c7d95047445ee9d44640b5372e0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:06:55.514352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-868fb7f6-efa0-4b76-9337-ec0ecd685bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:55.521216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1155/2275] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:06:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:06:55.829017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:06:55.830494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:55.843284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d4af8-39b2-49b4-a456-27f2e4475b6d service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:06:55.846867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:06:55.847480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:55.847817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:55.848091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:55.849794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:55.859227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:06:55.860673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:06:55.869038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1121/2276] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:06:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:06:55.991952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71654e65-4025-4b8c-a11e-00bec57a67b1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:06:55.992327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71654e65-4025-4b8c-a11e-00bec57a67b1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:06:55.992643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71654e65-4025-4b8c-a11e-00bec57a67b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:06:56.013758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-71654e65-4025-4b8c-a11e-00bec57a67b1 None None] MFA Rules not processed for user `d4d85df24c9e4312bbe2da197f55e116`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:06:56.014901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-71654e65-4025-4b8c-a11e-00bec57a67b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:06:56.078083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1156/2277] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:06:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:07:05.865030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=40 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:07:06.065446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=31 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:07:36.821631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:36.830908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c65925-5316-49a9-9b57-d3301813c918 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:36.834546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:36.835160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:36.835445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:36.835750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:36.837499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:36.901049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:36.902502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:36.912495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1122/2278] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:07:40.439820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71654e65-4025-4b8c-a11e-00bec57a67b1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:40.451666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71654e65-4025-4b8c-a11e-00bec57a67b1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:40.456599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:40.462162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:40.462162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:40.462162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:40.462162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:40.471649 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:40.473246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:40.484099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1157/2279] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:07:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:07:40.561506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:40.561506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:40.573957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88741cdf-dc81-4bf9-8037-5b506fb22ca3 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:40.578587 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:40.579226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:40.579506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:40.579817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] PATH_INFO: `/v3/users/972bc4e9bc9e49b4a3bc2a693b05e4ab` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:40.580970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=972bc4e9bc9e49b4a3bc2a693b05e4ab)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:40.583162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:40.743631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972bc4e9bc9e49b4a3bc2a693b05e4ab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:40.751209 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972bc4e9bc9e49b4a3bc2a693b05e4ab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:40.809537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972bc4e9bc9e49b4a3bc2a693b05e4ab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:40.810140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972bc4e9bc9e49b4a3bc2a693b05e4ab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:40.822698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13effc24-6a64-4ac3-b1d1-5584301f1b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972bc4e9bc9e49b4a3bc2a693b05e4ab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:40.828792 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1123/2280] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:07:40 2022] DELETE /identity/v3/users/972bc4e9bc9e49b4a3bc2a693b05e4ab => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:07:41.060588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:41.074460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b1de9-6b44-4a9e-8b8f-b4571d209c74 service placement] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:41.134179 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:41.135556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.136215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.136408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] PATH_INFO: `/v3/projects/aae673506a1a4ea2b3209c8a86b3a33b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.137587 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae673506a1a4ea2b3209c8a86b3a33b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:41.139582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:41.233953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae673506a1a4ea2b3209c8a86b3a33b', 'request_id': 'req-638d715e-ace5-4000-8d4b-3e8cbd80eac8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:41.301096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae673506a1a4ea2b3209c8a86b3a33b', 'request_id': 'req-638d715e-ace5-4000-8d4b-3e8cbd80eac8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:41.307451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae673506a1a4ea2b3209c8a86b3a33b', 'request_id': 'req-638d715e-ace5-4000-8d4b-3e8cbd80eac8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:41.318656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] The token cache is being invalidate because project aae673506a1a4ea2b3209c8a86b3a33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:07:41.318890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:41.319550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-638d715e-ace5-4000-8d4b-3e8cbd80eac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:07:41.321101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1158/2281] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:07:41 2022] DELETE /identity/v3/projects/aae673506a1a4ea2b3209c8a86b3a33b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:07:41.326747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd523f-fa83-4cb8-b027-27caaff7aff1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.327051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd523f-fa83-4cb8-b027-27caaff7aff1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.331048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd523f-fa83-4cb8-b027-27caaff7aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.347945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f2bd523f-fa83-4cb8-b027-27caaff7aff1 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:07:41.349217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f2bd523f-fa83-4cb8-b027-27caaff7aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:41.409645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1124/2282] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:07:41.418060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619e2a-baa7-40dc-9627-b9a9e737eb01 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.418348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619e2a-baa7-40dc-9627-b9a9e737eb01 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.418627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619e2a-baa7-40dc-9627-b9a9e737eb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.439058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f2619e2a-baa7-40dc-9627-b9a9e737eb01 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:07:41.440282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f2619e2a-baa7-40dc-9627-b9a9e737eb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:41.448435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1159/2283] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:07:41.452901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd523f-fa83-4cb8-b027-27caaff7aff1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:41.462705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd523f-fa83-4cb8-b027-27caaff7aff1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:41.467371 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:41.468020 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.471250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.471250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.471250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:07:41.471250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:41.471250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:41.472738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:07:41.484012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1125/2284] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:07:41.488249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619e2a-baa7-40dc-9627-b9a9e737eb01 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:41.498670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619e2a-baa7-40dc-9627-b9a9e737eb01 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:41.503748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:41.504372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.504656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.504932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.505635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:41.507594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:41.610655 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1160/2285] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:07:41 2022] POST /identity/v3/projects => generated 357 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:07:41.614694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:41.624869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6c523-b435-4096-ba77-c10bb9bad9cd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:41.680601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:41.681215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.681537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.682017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.682723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:41.684709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:41.781987 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1126/2286] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:07:41 2022] POST /identity/v3/users => generated 279 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:07:41.786182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:41.797477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3c154-7c28-425a-81dc-847cb6051f9c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:41.803995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:41.803995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.803995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.803995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.804380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:41.805817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:41.816675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1161/2287] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:07:41.820634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:41.831163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530804f-45d2-4336-9178-f3acafa25b81 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:41.835918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dee34-a291-4e80-89a5-ed7d28df83c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:41.836516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dee34-a291-4e80-89a5-ed7d28df83c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.836803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dee34-a291-4e80-89a5-ed7d28df83c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.837078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dee34-a291-4e80-89a5-ed7d28df83c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.838256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dee34-a291-4e80-89a5-ed7d28df83c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:41.839878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dee34-a291-4e80-89a5-ed7d28df83c5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:41.850619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1127/2288] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:07:41.855515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:41.865456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b09ea-5c50-418d-9c61-fa99db0e743a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:41.870159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:41.870814 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.871097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.871373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] PATH_INFO: `/v3/projects/6d25256994b642f0b3d8818f36c09899/users/75f1130fdce344edb4ebce3e5009e201/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.886182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d25256994b642f0b3d8818f36c09899, user_id=75f1130fdce344edb4ebce3e5009e201, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:41.888896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:41.946197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1162/2289] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:07:41 2022] PUT /identity/v3/projects/6d25256994b642f0b3d8818f36c09899/users/75f1130fdce344edb4ebce3e5009e201/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:07:41.958335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecb7d5-5112-4fbe-bb06-9fdc1a84b6a5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:41.958630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecb7d5-5112-4fbe-bb06-9fdc1a84b6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:41.958911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecb7d5-5112-4fbe-bb06-9fdc1a84b6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:41.986808 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d1ecb7d5-5112-4fbe-bb06-9fdc1a84b6a5 None None] MFA Rules not processed for user `75f1130fdce344edb4ebce3e5009e201`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:07:41.988072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d1ecb7d5-5112-4fbe-bb06-9fdc1a84b6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:42.050762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1128/2290] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:07:41 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:07:42.055279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:42.069596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:07:42.118691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93113a2-fe4c-4f2a-a0e8-eb8fde4bf147 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:42.123507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:42.124158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.124447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.124729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.125395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:42.127326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:42.218486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1163/2291] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:07:42 2022] POST /identity/v3/projects => generated 357 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:07:42.221937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecb7d5-5112-4fbe-bb06-9fdc1a84b6a5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:42.233353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecb7d5-5112-4fbe-bb06-9fdc1a84b6a5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:42.294713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:42.295343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.295675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.295957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.296644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:42.298875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:42.308902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:07:42.375745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1129/2292] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:07:42 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:07:42.380157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:42.392494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d94ef-2a8a-466f-ab4f-d0ddaeb3ffbc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:42.397327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:42.397947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.398236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.398551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.399270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:42.401002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:42.411823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1164/2293] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:07:42.416129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:42.426303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061794a-732d-4ce6-a3c1-b9fa4f9ce117 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:42.430981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:42.432378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.432671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.432954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] PATH_INFO: `/v3/projects/1c9c2d9e64474b6f95d37bc8a24b2812/users/c82795cce82e4987bc7725862d5aa352/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.448544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9c2d9e64474b6f95d37bc8a24b2812, user_id=c82795cce82e4987bc7725862d5aa352, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:42.451492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:42.512944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1130/2294] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:07:42 2022] PUT /identity/v3/projects/1c9c2d9e64474b6f95d37bc8a24b2812/users/c82795cce82e4987bc7725862d5aa352/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:07:42.516992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:42.527630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d80de-f697-487b-9861-4ba203a0472d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:42.585090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d3160-4ef1-4f32-837d-8070098d8171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:42.585712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d3160-4ef1-4f32-837d-8070098d8171 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.586001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d3160-4ef1-4f32-837d-8070098d8171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.586282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d3160-4ef1-4f32-837d-8070098d8171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.586980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d3160-4ef1-4f32-837d-8070098d8171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:42.588664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d3160-4ef1-4f32-837d-8070098d8171 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:42.598301 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1165/2295] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:07:42.602440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:42.613158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc11fe-7337-4b6a-bc01-369ce6e75081 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:42.617865 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:42.618536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.618827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.619679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] PATH_INFO: `/v3/projects/1c9c2d9e64474b6f95d37bc8a24b2812/users/c82795cce82e4987bc7725862d5aa352/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.621803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9c2d9e64474b6f95d37bc8a24b2812, user_id=c82795cce82e4987bc7725862d5aa352, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:42.624485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:42.635440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1131/2296] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:07:42 2022] PUT /identity/v3/projects/1c9c2d9e64474b6f95d37bc8a24b2812/users/c82795cce82e4987bc7725862d5aa352/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:07:42.645478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f2a05-6781-452d-a46c-fbb1f7a13550 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.645770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f2a05-6781-452d-a46c-fbb1f7a13550 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.646048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f2a05-6781-452d-a46c-fbb1f7a13550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.674115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-392f2a05-6781-452d-a46c-fbb1f7a13550 None None] MFA Rules not processed for user `c82795cce82e4987bc7725862d5aa352`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:07:42.675310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-392f2a05-6781-452d-a46c-fbb1f7a13550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:42.740739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1166/2297] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:07:42 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:07:42.752511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:42.754214 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:07:42.815961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04c069-4660-4584-9120-073496d595b0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:42.819653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:42.820276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:42.820563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:42.820838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:42.830060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:42.831494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:42.841114 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1132/2298] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:07:42 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:07:43.062196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f2a05-6781-452d-a46c-fbb1f7a13550 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:07:43.071672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f2a05-6781-452d-a46c-fbb1f7a13550 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:07:43.075632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288c1fa-e039-486b-b91b-591a5a5b263a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:07:43.076239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288c1fa-e039-486b-b91b-591a5a5b263a service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:07:43.076520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288c1fa-e039-486b-b91b-591a5a5b263a service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:07:43.076794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288c1fa-e039-486b-b91b-591a5a5b263a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:07:43.081872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:07:43.135476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288c1fa-e039-486b-b91b-591a5a5b263a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:07:43.137098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288c1fa-e039-486b-b91b-591a5a5b263a service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:07:43.145854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1167/2299] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:07:43 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:07:52.836733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:07:53.169577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=24 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:02.443454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:02.505069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db16091-1697-4284-a788-9f724cfe2db4 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:02.509766 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:02.510404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:02.510692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:02.510970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] PATH_INFO: `/v3/users/75f1130fdce344edb4ebce3e5009e201` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:02.512829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f1130fdce344edb4ebce3e5009e201)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:02.514746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:02.684511 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f1130fdce344edb4ebce3e5009e201'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:02.691487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f1130fdce344edb4ebce3e5009e201'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:02.784957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f1130fdce344edb4ebce3e5009e201'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:02.785588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f1130fdce344edb4ebce3e5009e201'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:02.798691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f1130fdce344edb4ebce3e5009e201'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:02.804710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1133/2300] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:02 2022] DELETE /identity/v3/users/75f1130fdce344edb4ebce3e5009e201 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:02.809901 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:02.810384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:02.810490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:02.830012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:02.831219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:02.888809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1168/2301] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:02.898424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:02.899859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:02.956938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957980eb-0aaa-454d-863b-349c799762a8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:02.960530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:02.961146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:02.961431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:02.961705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:02.970669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:02.972125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:02.981997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1134/2302] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:03.353677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:03.355109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:03.367251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acea044-018f-4a06-abb4-3e2b5ea3096e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:03.372590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:03.373245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:03.373561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:03.373845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] PATH_INFO: `/v3/projects/6d25256994b642f0b3d8818f36c09899` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:03.375036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d25256994b642f0b3d8818f36c09899)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:03.377005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:03.487411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d25256994b642f0b3d8818f36c09899', 'request_id': 'req-063854fa-a040-42d9-910e-7361f143a291'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.552321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d25256994b642f0b3d8818f36c09899', 'request_id': 'req-063854fa-a040-42d9-910e-7361f143a291'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.560254 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d25256994b642f0b3d8818f36c09899', 'request_id': 'req-063854fa-a040-42d9-910e-7361f143a291'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.572208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] The token cache is being invalidate because project 6d25256994b642f0b3d8818f36c09899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:03.572504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.573168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.575283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1169/2303] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:03 2022] DELETE /identity/v3/projects/6d25256994b642f0b3d8818f36c09899 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:03.579294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:03.589024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73c3fa-35cf-4404-924d-f4c18b777aa8 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:03.607972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:03.645581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:03.646249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:03.646533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:03.646806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] PATH_INFO: `/v3/users/c82795cce82e4987bc7725862d5aa352` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:03.648031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82795cce82e4987bc7725862d5aa352)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:03.649950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:03.866763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82795cce82e4987bc7725862d5aa352'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.876416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82795cce82e4987bc7725862d5aa352'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.950996 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82795cce82e4987bc7725862d5aa352'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.951891 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82795cce82e4987bc7725862d5aa352'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.966397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82795cce82e4987bc7725862d5aa352'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:03.972697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1135/2304] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:03 2022] DELETE /identity/v3/users/c82795cce82e4987bc7725862d5aa352 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:04.414788 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:04.416276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:04.433005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:04.434717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:04.497877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063854fa-a040-42d9-910e-7361f143a291 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:04.515782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e89ac4-45ad-41dc-9bf0-35487d8787d9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:04.517637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:04.520687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:04.521337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:04.521632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:04.523683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] PATH_INFO: `/v3/projects/1c9c2d9e64474b6f95d37bc8a24b2812` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:04.523683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9c2d9e64474b6f95d37bc8a24b2812)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:04.525621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:04.564169 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24a588-b33e-439d-9937-4810bcad62b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:04.564792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24a588-b33e-439d-9937-4810bcad62b1 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:04.565074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24a588-b33e-439d-9937-4810bcad62b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:04.565347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24a588-b33e-439d-9937-4810bcad62b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:04.567123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8e24a588-b33e-439d-9937-4810bcad62b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:04.625931 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24a588-b33e-439d-9937-4810bcad62b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:04.627784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24a588-b33e-439d-9937-4810bcad62b1 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:04.636892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1170/2305] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:04.638845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:04.642725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9c2d9e64474b6f95d37bc8a24b2812', 'request_id': 'req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:04.649220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9c2d9e64474b6f95d37bc8a24b2812', 'request_id': 'req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:04.654933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9c2d9e64474b6f95d37bc8a24b2812', 'request_id': 'req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:04.724167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] The token cache is being invalidate because project 1c9c2d9e64474b6f95d37bc8a24b2812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:04.724515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:04.725153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bc2c93ae-6148-4fd3-bdf2-4a0e52b8346e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:04.726666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1136/2306] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:04 2022] DELETE /identity/v3/projects/1c9c2d9e64474b6f95d37bc8a24b2812 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:04.732546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8357e4-447f-4a48-95c6-84c45cf3736e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:04.732840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8357e4-447f-4a48-95c6-84c45cf3736e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:04.733120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8357e4-447f-4a48-95c6-84c45cf3736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:04.753483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-db8357e4-447f-4a48-95c6-84c45cf3736e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:04.754701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-db8357e4-447f-4a48-95c6-84c45cf3736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:04.821639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1171/2307] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:04.829994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2acb-c99a-4094-ae98-d6e11aeb6a5f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:04.830299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2acb-c99a-4094-ae98-d6e11aeb6a5f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:04.830580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2acb-c99a-4094-ae98-d6e11aeb6a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:04.852182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-312e2acb-c99a-4094-ae98-d6e11aeb6a5f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:04.853405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-312e2acb-c99a-4094-ae98-d6e11aeb6a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:04.862618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1137/2308] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:04.866949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8357e4-447f-4a48-95c6-84c45cf3736e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:04.878068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8357e4-447f-4a48-95c6-84c45cf3736e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:04.882743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:04.883375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:04.887564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:04.887564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:04.887564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:08:04.887564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:04.887564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:04.899127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1172/2309] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:08:04.903654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e2acb-c99a-4094-ae98-d6e11aeb6a5f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:04.913482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e2acb-c99a-4094-ae98-d6e11aeb6a5f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:04.918669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:04.919288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:04.919682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:04.919958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:04.920684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:04.922584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.026352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1138/2310] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:04 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:08:05.032662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.042845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7b5a6-1eb0-4186-a230-2d280d5584a2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.088190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:05.100801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.101413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.101695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.101970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.102624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.104604 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.213980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1173/2311] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:05 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:08:05.217128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.227793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b5c17-a9b7-4598-bb85-697250f4ab76 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.233083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.233705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.233992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.234269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.235054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.236732 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.247938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1139/2312] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:05.251967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.262679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e72d38-ecc6-4972-85b6-d0ebd81d1dff admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.267643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf125-ad65-4d22-a108-e547413ab4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.268257 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf125-ad65-4d22-a108-e547413ab4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.268541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf125-ad65-4d22-a108-e547413ab4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.268816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf125-ad65-4d22-a108-e547413ab4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.269512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdf125-ad65-4d22-a108-e547413ab4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.271149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdf125-ad65-4d22-a108-e547413ab4b9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.282343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1174/2313] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:05.286535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.297208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726552e-3a51-4610-b8c0-4ba60c03f8eb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.302102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.302772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.303065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.303345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] PATH_INFO: `/v3/projects/04f56b08068a4082aa5e0c5f8a273f38/users/fcc9cd6b266b43a088d295111da86416/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.319119 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f56b08068a4082aa5e0c5f8a273f38, user_id=fcc9cd6b266b43a088d295111da86416, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.321837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.380963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1140/2314] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:05 2022] PUT /identity/v3/projects/04f56b08068a4082aa5e0c5f8a273f38/users/fcc9cd6b266b43a088d295111da86416/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:05.391193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd561cd2-cf67-4ee9-b2fc-afdfb6197dc2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.391490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd561cd2-cf67-4ee9-b2fc-afdfb6197dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.391891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd561cd2-cf67-4ee9-b2fc-afdfb6197dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.419954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cd561cd2-cf67-4ee9-b2fc-afdfb6197dc2 None None] MFA Rules not processed for user `fcc9cd6b266b43a088d295111da86416`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:05.421229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cd561cd2-cf67-4ee9-b2fc-afdfb6197dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:05.487765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1175/2315] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:05.492377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.561931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf899e84-4026-4e5d-88e8-7c44167a8bf4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.566590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.567215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.567495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.567796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.568453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.570334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.644120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1141/2316] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:05 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:05.645018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd561cd2-cf67-4ee9-b2fc-afdfb6197dc2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.656523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd561cd2-cf67-4ee9-b2fc-afdfb6197dc2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.716258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.717174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.717467 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.717747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.718433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.720418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.800708 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1176/2317] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:05 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:08:05.804654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.815326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97ca9-cd5d-40b4-9b50-5f3e58f2889e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.820192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.820825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.821115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.821394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.822148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.823773 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.835699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1142/2318] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:05.840550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.850777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6a92-9521-45cd-a565-ed3f5e27ba11 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:05.855615 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:05.856328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:05.856618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:05.856899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] PATH_INFO: `/v3/projects/d03637c8729a49d9b249e5a5fabdf9a7/users/f12137dbc35448dbbfd8b14a13d5757f/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:05.872826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03637c8729a49d9b249e5a5fabdf9a7, user_id=f12137dbc35448dbbfd8b14a13d5757f, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:05.875518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:05.983850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1177/2319] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:05 2022] PUT /identity/v3/projects/d03637c8729a49d9b249e5a5fabdf9a7/users/f12137dbc35448dbbfd8b14a13d5757f/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:05.988077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:05.998634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f589a-e7a5-4874-9725-1eefcded9f2a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:06.044096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:06.062755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420795b-a04e-4496-892a-0fe3d6408954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:06.063397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420795b-a04e-4496-892a-0fe3d6408954 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:06.063728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420795b-a04e-4496-892a-0fe3d6408954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:06.064010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420795b-a04e-4496-892a-0fe3d6408954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:06.064750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420795b-a04e-4496-892a-0fe3d6408954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:06.066422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420795b-a04e-4496-892a-0fe3d6408954 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:06.081270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1143/2320] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:06.082520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:06.093684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961217e-045a-4e50-96a5-8b03f0f55fec admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:06.098461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:06.099131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:06.099412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:06.099717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] PATH_INFO: `/v3/projects/d03637c8729a49d9b249e5a5fabdf9a7/users/f12137dbc35448dbbfd8b14a13d5757f/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:06.101658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03637c8729a49d9b249e5a5fabdf9a7, user_id=f12137dbc35448dbbfd8b14a13d5757f, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:06.104445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:06.115609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1178/2321] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:06 2022] PUT /identity/v3/projects/d03637c8729a49d9b249e5a5fabdf9a7/users/f12137dbc35448dbbfd8b14a13d5757f/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:06.125764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7ad2-6ac4-4df6-a322-925663c1835e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:06.126101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7ad2-6ac4-4df6-a322-925663c1835e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:06.126415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7ad2-6ac4-4df6-a322-925663c1835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:06.156537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-107f7ad2-6ac4-4df6-a322-925663c1835e None None] MFA Rules not processed for user `f12137dbc35448dbbfd8b14a13d5757f`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:06.157763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-107f7ad2-6ac4-4df6-a322-925663c1835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:06.226790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1144/2322] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:06.238824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:06.240630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:06.250460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:06.299603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8885-e8b8-49f3-83c2-630a05850e4a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:06.303219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd5d9f-29b1-45f8-a169-66bcc2753890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:06.303872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd5d9f-29b1-45f8-a169-66bcc2753890 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:06.304161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd5d9f-29b1-45f8-a169-66bcc2753890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:06.304441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd5d9f-29b1-45f8-a169-66bcc2753890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:06.314033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd5d9f-29b1-45f8-a169-66bcc2753890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:06.315487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd5d9f-29b1-45f8-a169-66bcc2753890 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:06.324952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1179/2323] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:06.353545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7ad2-6ac4-4df6-a322-925663c1835e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:06.414866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7ad2-6ac4-4df6-a322-925663c1835e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:06.420496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:06.421146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:06.421434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:06.421711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] PATH_INFO: `/v3/users/fcc9cd6b266b43a088d295111da86416` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:06.422885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc9cd6b266b43a088d295111da86416)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:06.425104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:06.600629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc9cd6b266b43a088d295111da86416'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:06.607867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc9cd6b266b43a088d295111da86416'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:06.678469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc9cd6b266b43a088d295111da86416'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:06.679226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc9cd6b266b43a088d295111da86416'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:06.681487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:06.692359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc9cd6b266b43a088d295111da86416'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:06.698539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1145/2324] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:06 2022] DELETE /identity/v3/users/fcc9cd6b266b43a088d295111da86416 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:06.704063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:06.705619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:06.705619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:06.724605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:06.725832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:06.788626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1180/2325] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:06.799066 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:06.800514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:06.859892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e0d6f-e420-4cc4-927d-560ae3ec08e0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:06.863738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:06.864364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:06.864651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:06.864931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:06.874060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:06.875495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:06.885593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1146/2326] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:07.392058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:07.393513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:07.406430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703a5c6-8205-4e91-aae1-bdcdd878e91b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:07.411182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:07.411876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:07.412164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:07.412453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] PATH_INFO: `/v3/projects/04f56b08068a4082aa5e0c5f8a273f38` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:07.413641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f56b08068a4082aa5e0c5f8a273f38)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:07.415911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:07.553363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f56b08068a4082aa5e0c5f8a273f38', 'request_id': 'req-b77c17ea-d03a-4b2c-98a3-615decfb43e5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.644383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f56b08068a4082aa5e0c5f8a273f38', 'request_id': 'req-b77c17ea-d03a-4b2c-98a3-615decfb43e5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.651823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f56b08068a4082aa5e0c5f8a273f38', 'request_id': 'req-b77c17ea-d03a-4b2c-98a3-615decfb43e5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.665756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] The token cache is being invalidate because project 04f56b08068a4082aa5e0c5f8a273f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:07.666236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.666888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.669419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1181/2327] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:07 2022] DELETE /identity/v3/projects/04f56b08068a4082aa5e0c5f8a273f38 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:07.673647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:07.684538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83955fc3-a04d-475c-97c8-e9d3c33da58a service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:07.755401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:07.755401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:07.755401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:07.755401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] PATH_INFO: `/v3/users/f12137dbc35448dbbfd8b14a13d5757f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:07.755968 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12137dbc35448dbbfd8b14a13d5757f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:07.756890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:07.898697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12137dbc35448dbbfd8b14a13d5757f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.906086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12137dbc35448dbbfd8b14a13d5757f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.969599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12137dbc35448dbbfd8b14a13d5757f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.972896 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12137dbc35448dbbfd8b14a13d5757f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.985042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3e209cc1-b6ba-46a9-9ecc-79314208849e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12137dbc35448dbbfd8b14a13d5757f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:07.991731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1147/2328] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:07 2022] DELETE /identity/v3/users/f12137dbc35448dbbfd8b14a13d5757f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:08.446955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:08.448997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:08.517258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:08.567579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c17ea-d03a-4b2c-98a3-615decfb43e5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:08.629350 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:08.630024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:08.630315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:08.630599 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] PATH_INFO: `/v3/projects/d03637c8729a49d9b249e5a5fabdf9a7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:08.631886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03637c8729a49d9b249e5a5fabdf9a7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:08.633910 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:08.794441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03637c8729a49d9b249e5a5fabdf9a7', 'request_id': 'req-994a3bf5-4984-4fb3-adb6-e5304fece946'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:08.866906 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03637c8729a49d9b249e5a5fabdf9a7', 'request_id': 'req-994a3bf5-4984-4fb3-adb6-e5304fece946'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:08.873884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03637c8729a49d9b249e5a5fabdf9a7', 'request_id': 'req-994a3bf5-4984-4fb3-adb6-e5304fece946'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:08.885962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] The token cache is being invalidate because project d03637c8729a49d9b249e5a5fabdf9a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:08.886263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:08.887116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-994a3bf5-4984-4fb3-adb6-e5304fece946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:08.888894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1182/2329] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:08 2022] DELETE /identity/v3/projects/d03637c8729a49d9b249e5a5fabdf9a7 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:08.894055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e66906-11a4-4b6d-9a84-19f3dc23c5f3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:08.900246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e66906-11a4-4b6d-9a84-19f3dc23c5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:08.900246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e66906-11a4-4b6d-9a84-19f3dc23c5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:08.901719 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:08.985405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f2e66906-11a4-4b6d-9a84-19f3dc23c5f3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:08.986832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f2e66906-11a4-4b6d-9a84-19f3dc23c5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:09.063631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1148/2330] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:09.072638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312fdc5-285e-4e7c-8f36-bc7e5ed4aeb3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.075264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312fdc5-285e-4e7c-8f36-bc7e5ed4aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.075264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312fdc5-285e-4e7c-8f36-bc7e5ed4aeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.098110 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4312fdc5-285e-4e7c-8f36-bc7e5ed4aeb3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:09.099394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4312fdc5-285e-4e7c-8f36-bc7e5ed4aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:09.108532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1183/2331] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:09.112889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e66906-11a4-4b6d-9a84-19f3dc23c5f3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:09.125881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e66906-11a4-4b6d-9a84-19f3dc23c5f3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:09.133739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:09.134645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.136200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.137211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.138160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:08:09.138696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:09.142547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:09.157323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1149/2332] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:08:09.161879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312fdc5-285e-4e7c-8f36-bc7e5ed4aeb3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:09.171776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312fdc5-285e-4e7c-8f36-bc7e5ed4aeb3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:09.178795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:09.179508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.179899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.180279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.181167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:09.183283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:09.273188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1184/2333] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:09 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:09.274559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:09.285525 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad6993-82ba-4011-bac4-a4454011b291 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:09.356446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:09.357074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.357355 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.357628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.358291 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:09.360261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:09.472572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1150/2334] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:09 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:09.473775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:09.485196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ec329-0992-4d74-b9c2-32a79b95cbcf admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:09.490213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:09.491018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.491274 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.491627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.492354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:09.493974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:09.507769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1185/2335] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:09.512349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:09.523086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9105c81-7a5d-4a71-88ec-38984abe00cc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:09.528194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e42826-ac04-4a2d-adff-5ed744d54343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:09.528857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e42826-ac04-4a2d-adff-5ed744d54343 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.529146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e42826-ac04-4a2d-adff-5ed744d54343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.529426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e42826-ac04-4a2d-adff-5ed744d54343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.530223 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e42826-ac04-4a2d-adff-5ed744d54343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:09.531782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e42826-ac04-4a2d-adff-5ed744d54343 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:09.551084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1151/2336] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:09.552463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:09.563313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b210e-1a1c-4e8e-bba2-c85c09aa1279 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:09.567963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:09.568638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.568923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.569201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] PATH_INFO: `/v3/projects/1e3bfd786f6f4c6c82eb51bd08840987/users/a437c9779a044579978fcf84429bb41f/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.584317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3bfd786f6f4c6c82eb51bd08840987, user_id=a437c9779a044579978fcf84429bb41f, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:09.586979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:09.707688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:09.715094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1186/2337] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:09 2022] PUT /identity/v3/projects/1e3bfd786f6f4c6c82eb51bd08840987/users/a437c9779a044579978fcf84429bb41f/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:08:09.731961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03882-c91d-4312-9920-4eb95a3de85d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.732336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03882-c91d-4312-9920-4eb95a3de85d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.732638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03882-c91d-4312-9920-4eb95a3de85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.774861 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3ea03882-c91d-4312-9920-4eb95a3de85d None None] MFA Rules not processed for user `a437c9779a044579978fcf84429bb41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:09.778389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3ea03882-c91d-4312-9920-4eb95a3de85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:09.807204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:09.844024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1152/2338] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:09.848544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:09.915321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4723fa-8f2e-4f50-bdca-0d114d71a1ec admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:09.921618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:09.922505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:09.922786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:09.923052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:09.924148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:09.926779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:10.009806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1187/2339] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:09 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:10.014090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03882-c91d-4312-9920-4eb95a3de85d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:10.025191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03882-c91d-4312-9920-4eb95a3de85d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:10.085135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:10.085766 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.086150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.086479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:10.087136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:10.089271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:10.225952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1153/2340] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:10 2022] POST /identity/v3/users => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:10.227099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:10.238851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc459d-4921-48bf-bf1f-572b271c63d7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:10.246727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:10.250646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.250646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.250646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:10.250646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:10.250646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:10.263981 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1188/2341] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:10.268038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:10.280191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aac06-f44e-4695-b174-e64db4c06a45 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:10.285059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:10.285730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.286014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.286288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] PATH_INFO: `/v3/projects/94cc49d312f144248362925015193a8c/users/7dc0ff9c1a73449fa019456e58af8cc9/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:10.302625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cc49d312f144248362925015193a8c, user_id=7dc0ff9c1a73449fa019456e58af8cc9, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:10.305333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:10.393709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1154/2342] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:10 2022] PUT /identity/v3/projects/94cc49d312f144248362925015193a8c/users/7dc0ff9c1a73449fa019456e58af8cc9/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:10.397892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:10.409192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a6d70-355c-4aa2-baba-d8efc977b67b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:10.469764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce11a30-e355-49f3-a9d6-d1bad3bbb97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:10.470423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11a30-e355-49f3-a9d6-d1bad3bbb97e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.470712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11a30-e355-49f3-a9d6-d1bad3bbb97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.471397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11a30-e355-49f3-a9d6-d1bad3bbb97e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:10.472124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce11a30-e355-49f3-a9d6-d1bad3bbb97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:10.473753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce11a30-e355-49f3-a9d6-d1bad3bbb97e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:10.484378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1189/2343] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:10.488597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:10.499000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89fc7-1ead-4d9f-822c-a8249ae7aa56 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:10.503714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:10.504388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.504680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.504953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] PATH_INFO: `/v3/projects/94cc49d312f144248362925015193a8c/users/7dc0ff9c1a73449fa019456e58af8cc9/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:10.506910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cc49d312f144248362925015193a8c, user_id=7dc0ff9c1a73449fa019456e58af8cc9, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:10.509572 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:10.530186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1155/2344] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:10 2022] PUT /identity/v3/projects/94cc49d312f144248362925015193a8c/users/7dc0ff9c1a73449fa019456e58af8cc9/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:10.532347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.532756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.532893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:10.563093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] MFA Rules not processed for user `7dc0ff9c1a73449fa019456e58af8cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:10.564439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:10.659488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:10.670917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1190/2345] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:10.682399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:10.686606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:10.829088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8cc6f-5d6b-4a8a-97e9-168025fcaff7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:10.829088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2984570-c827-460c-979b-c940c75e9400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:10.829088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2984570-c827-460c-979b-c940c75e9400 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.829088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2984570-c827-460c-979b-c940c75e9400 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.830337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2984570-c827-460c-979b-c940c75e9400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:10.846852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2984570-c827-460c-979b-c940c75e9400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:10.848478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2984570-c827-460c-979b-c940c75e9400 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:10.861050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1156/2346] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:10.924306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:10.925574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:10.939430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787ebfa-8d81-414d-a88e-29960133ee20 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:10.943509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:10.944166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:10.944456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:10.944733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:11.041832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:11.043354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:11.054856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1191/2347] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:14.400005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410caa-3f7d-4fac-9b09-80a87742b1da None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:14.400508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410caa-3f7d-4fac-9b09-80a87742b1da None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:14.400604 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410caa-3f7d-4fac-9b09-80a87742b1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:14.420399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-29410caa-3f7d-4fac-9b09-80a87742b1da None None] MFA Rules not processed for user `f030155981d741f1a42b9b84ca5d6af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:14.421612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-29410caa-3f7d-4fac-9b09-80a87742b1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:14.425876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:14.477570 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1157/2348] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:16.043144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:16.103226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43030ed-28d9-4c42-9b13-02778a406918 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:16.103813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43030ed-28d9-4c42-9b13-02778a406918 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:16.103902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43030ed-28d9-4c42-9b13-02778a406918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:16.116023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c144fa-7ec8-4cc2-b907-40b075114344 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:16.121201 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:16.121986 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:16.122304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:16.122587 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] PATH_INFO: `/v3/users/a437c9779a044579978fcf84429bb41f` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:16.123864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a437c9779a044579978fcf84429bb41f)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:16.125047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f43030ed-28d9-4c42-9b13-02778a406918 None None] MFA Rules not processed for user `14e5f1b4073742b4b9bd8a48cce38341`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:16.125919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:16.126561 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f43030ed-28d9-4c42-9b13-02778a406918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:16.187775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1158/2349] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:08:16.273936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a437c9779a044579978fcf84429bb41f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:16.374004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a437c9779a044579978fcf84429bb41f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:16.384862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a437c9779a044579978fcf84429bb41f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:16.385586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a437c9779a044579978fcf84429bb41f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:16.386791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a437c9779a044579978fcf84429bb41f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:16.408115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1192/2350] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:16 2022] DELETE /identity/v3/users/a437c9779a044579978fcf84429bb41f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:16.412960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:16.413408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:16.413702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:16.447798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:16.447798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:16.520533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1159/2351] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:16.530519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:16.532096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:16.546576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:16.597761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351105b-1a4b-4c83-845b-beadf8e0f204 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:16.602376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:16.603140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:16.603494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:16.603888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:16.613885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:16.615454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:16.627163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1193/2352] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:17.068473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:17.069889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:17.101082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605906c-8678-4fd6-9180-cd9d65fae88f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:17.101082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:17.101082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:17.101082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:17.101822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] PATH_INFO: `/v3/projects/1e3bfd786f6f4c6c82eb51bd08840987` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:17.102896 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3bfd786f6f4c6c82eb51bd08840987)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:17.106336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:17.236643 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3bfd786f6f4c6c82eb51bd08840987', 'request_id': 'req-fac37694-8238-4006-91a6-826b15f6518b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.248225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3bfd786f6f4c6c82eb51bd08840987', 'request_id': 'req-fac37694-8238-4006-91a6-826b15f6518b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.249621 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:17.255164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3bfd786f6f4c6c82eb51bd08840987', 'request_id': 'req-fac37694-8238-4006-91a6-826b15f6518b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.346501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] The token cache is being invalidate because project 1e3bfd786f6f4c6c82eb51bd08840987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:17.346815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.347472 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.349180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1160/2353] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:17 2022] DELETE /identity/v3/projects/1e3bfd786f6f4c6c82eb51bd08840987 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:17.353226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:17.364082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81f24-0719-4989-9654-e7f94888e01a service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:17.428415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:17.429157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:17.429538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:17.429918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] PATH_INFO: `/v3/users/7dc0ff9c1a73449fa019456e58af8cc9` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:17.431343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc0ff9c1a73449fa019456e58af8cc9)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:17.433979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:17.617247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc0ff9c1a73449fa019456e58af8cc9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.712810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc0ff9c1a73449fa019456e58af8cc9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.720977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc0ff9c1a73449fa019456e58af8cc9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.721654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc0ff9c1a73449fa019456e58af8cc9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.722965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24465a0b-0294-47d0-8398-20ee43a82496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc0ff9c1a73449fa019456e58af8cc9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:17.755939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1194/2354] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:17 2022] DELETE /identity/v3/users/7dc0ff9c1a73449fa019456e58af8cc9 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:18.252497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:18.253966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:18.294260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac37694-8238-4006-91a6-826b15f6518b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:18.370377 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:18.371860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:18.372225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:18.372563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] PATH_INFO: `/v3/projects/94cc49d312f144248362925015193a8c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:18.373981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cc49d312f144248362925015193a8c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:18.376218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:18.503266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cc49d312f144248362925015193a8c', 'request_id': 'req-28d7bd36-b26e-4c92-b664-2381229867ff'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:18.509447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cc49d312f144248362925015193a8c', 'request_id': 'req-28d7bd36-b26e-4c92-b664-2381229867ff'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:18.515820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cc49d312f144248362925015193a8c', 'request_id': 'req-28d7bd36-b26e-4c92-b664-2381229867ff'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:18.598232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] The token cache is being invalidate because project 94cc49d312f144248362925015193a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:18.598634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:18.599322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-28d7bd36-b26e-4c92-b664-2381229867ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:18.601097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1161/2355] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:18 2022] DELETE /identity/v3/projects/94cc49d312f144248362925015193a8c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:18.606717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb489bd-6b4f-4495-88af-76e23111c25d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:18.607197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb489bd-6b4f-4495-88af-76e23111c25d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:18.607398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb489bd-6b4f-4495-88af-76e23111c25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:18.627949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bcb489bd-6b4f-4495-88af-76e23111c25d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:18.629340 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bcb489bd-6b4f-4495-88af-76e23111c25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:18.660997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:18.695959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1195/2356] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:18.705325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d3c1a-82a9-4ff8-a811-1b0bd02b9c70 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:18.705628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d3c1a-82a9-4ff8-a811-1b0bd02b9c70 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:18.705880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d3c1a-82a9-4ff8-a811-1b0bd02b9c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:18.728083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7d2d3c1a-82a9-4ff8-a811-1b0bd02b9c70 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:18.729807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7d2d3c1a-82a9-4ff8-a811-1b0bd02b9c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:18.738902 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1162/2357] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:08:18.743411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb489bd-6b4f-4495-88af-76e23111c25d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:18.755821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb489bd-6b4f-4495-88af-76e23111c25d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:18.761172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:18.762609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:18.763068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:18.763411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:18.764216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:08:18.764715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:18.766436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:18.782593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1196/2358] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:08:18.784244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d3c1a-82a9-4ff8-a811-1b0bd02b9c70 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:18.797271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d3c1a-82a9-4ff8-a811-1b0bd02b9c70 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:18.804123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:18.805475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:18.805867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:18.806695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:18.807507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:18.810982 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:18.970245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1163/2359] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:18 2022] POST /identity/v3/projects => generated 407 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:18.971963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:18.982554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119790f-978c-474a-84b1-18c859295407 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.049381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.050079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.050389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.050659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.051335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.053485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:19.206767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:19.208008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1197/2360] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:18 2022] POST /identity/v3/users => generated 304 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:19.217469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126077-df2f-45d8-8761-ef74c209835a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.222161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.222787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.223072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.223349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.224082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.225679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:19.237534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1164/2361] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:08:19.241756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:19.252530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2ae43-45ed-46a9-b2c6-9995b8ca8822 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.258518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec2f0d-5b19-464c-acba-e41074dbf40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.259235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec2f0d-5b19-464c-acba-e41074dbf40e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.259628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec2f0d-5b19-464c-acba-e41074dbf40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.259987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec2f0d-5b19-464c-acba-e41074dbf40e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.261907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec2f0d-5b19-464c-acba-e41074dbf40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.263802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec2f0d-5b19-464c-acba-e41074dbf40e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:19.275304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1198/2362] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:19.280129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:19.291230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeabd8-25a1-4bf5-a362-ced1f2389e08 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.298290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.299248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.299550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.302321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] PATH_INFO: `/v3/projects/9e7025a5de6c4cee87b03c401fca632d/users/c4035f0c1c5d45b198c1e16558a2fa2c/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.324276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7025a5de6c4cee87b03c401fca632d, user_id=c4035f0c1c5d45b198c1e16558a2fa2c, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.327307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:19.397694 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1165/2363] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:19 2022] PUT /identity/v3/projects/9e7025a5de6c4cee87b03c401fca632d/users/c4035f0c1c5d45b198c1e16558a2fa2c/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:19.409296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216130a-a71d-4ed8-b624-c5cb2ef195f3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.409592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216130a-a71d-4ed8-b624-c5cb2ef195f3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.409880 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216130a-a71d-4ed8-b624-c5cb2ef195f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.439922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6216130a-a71d-4ed8-b624-c5cb2ef195f3 None None] MFA Rules not processed for user `c4035f0c1c5d45b198c1e16558a2fa2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:19.441157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6216130a-a71d-4ed8-b624-c5cb2ef195f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:19.509789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1199/2364] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:19 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:19.515424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:19.528576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:19.588875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040c006-2f08-4de8-a079-f6eac23bc9a3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.598014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.599399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.599745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.600026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.600730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.603360 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:19.704141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1166/2365] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:19 2022] POST /identity/v3/projects => generated 409 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:19.707813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216130a-a71d-4ed8-b624-c5cb2ef195f3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:19.721505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216130a-a71d-4ed8-b624-c5cb2ef195f3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.749299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:19.786595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.787148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.787439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.787750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.788437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.790420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:19.876902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1200/2366] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:19 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:19.879562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:19.894013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b179268-ce84-41fb-8ab1-367b28d034cf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.900287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.901282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.901467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.901740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.902798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.904705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:19.918358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1167/2367] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:19.923045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:19.936197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a065c-c1c7-4e18-baa8-d172085a8ab6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:19.942430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:19.944066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:19.944553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:19.945153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] PATH_INFO: `/v3/projects/55cfa24bbbf64ce9b55dfafb01e3128c/users/d830544d175b40fda3b61e76482a9cd9/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:19.965683 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cfa24bbbf64ce9b55dfafb01e3128c, user_id=d830544d175b40fda3b61e76482a9cd9, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:19.968256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:20.065717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1201/2368] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:19 2022] PUT /identity/v3/projects/55cfa24bbbf64ce9b55dfafb01e3128c/users/d830544d175b40fda3b61e76482a9cd9/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:20.069675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:20.082544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f43d-76ab-4f9b-a14e-21804c8aecd1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:20.145238 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a62a89-2153-4771-abf7-fa794f596e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:20.145862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62a89-2153-4771-abf7-fa794f596e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:20.146142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62a89-2153-4771-abf7-fa794f596e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:20.146459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62a89-2153-4771-abf7-fa794f596e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:20.147132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a62a89-2153-4771-abf7-fa794f596e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:20.148779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a62a89-2153-4771-abf7-fa794f596e9e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:20.159802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1168/2369] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:08:20.164063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:20.178311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f402a-436e-41ed-b3ff-8f669c4f2468 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:20.184749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:20.186000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:20.186296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:20.186572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] PATH_INFO: `/v3/projects/55cfa24bbbf64ce9b55dfafb01e3128c/users/d830544d175b40fda3b61e76482a9cd9/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:20.189115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cfa24bbbf64ce9b55dfafb01e3128c, user_id=d830544d175b40fda3b61e76482a9cd9, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:20.192045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:20.205389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1202/2370] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:20 2022] PUT /identity/v3/projects/55cfa24bbbf64ce9b55dfafb01e3128c/users/d830544d175b40fda3b61e76482a9cd9/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:20.275756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863cd70-005d-49e4-8894-39cafe291f80 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:20.286115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863cd70-005d-49e4-8894-39cafe291f80 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:20.286115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863cd70-005d-49e4-8894-39cafe291f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:20.310759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5863cd70-005d-49e4-8894-39cafe291f80 None None] MFA Rules not processed for user `d830544d175b40fda3b61e76482a9cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:20.312109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5863cd70-005d-49e4-8894-39cafe291f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:20.398338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1169/2371] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:20 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:20.410567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:20.412544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:20.479484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877408b-ed8b-48de-9e14-cafb906c1fe2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:20.483631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:20.484267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:20.486701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:20.486701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:20.500742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:20.502184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:20.515183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1203/2372] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:20.900134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863cd70-005d-49e4-8894-39cafe291f80 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:20.906335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:20.910517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863cd70-005d-49e4-8894-39cafe291f80 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:20.914228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eaeb53-83ab-4c91-a5af-23d9085f0459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:20.918160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaeb53-83ab-4c91-a5af-23d9085f0459 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:20.918160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaeb53-83ab-4c91-a5af-23d9085f0459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:20.918160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaeb53-83ab-4c91-a5af-23d9085f0459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:20.974930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eaeb53-83ab-4c91-a5af-23d9085f0459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:20.976451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eaeb53-83ab-4c91-a5af-23d9085f0459 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:20.987010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1170/2373] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:20 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:30.510992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:31.101549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=26 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:31.699424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:31.772945 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adbac-e29c-4796-92ba-7a9c120100c9 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:31.777685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:31.778641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:31.778909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:31.779282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] PATH_INFO: `/v3/users/c4035f0c1c5d45b198c1e16558a2fa2c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:31.780516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4035f0c1c5d45b198c1e16558a2fa2c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:31.782454 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:31.958664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4035f0c1c5d45b198c1e16558a2fa2c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.030461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4035f0c1c5d45b198c1e16558a2fa2c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.037672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4035f0c1c5d45b198c1e16558a2fa2c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.038304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4035f0c1c5d45b198c1e16558a2fa2c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.038923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4035f0c1c5d45b198c1e16558a2fa2c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.057283 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1204/2374] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:31 2022] DELETE /identity/v3/users/c4035f0c1c5d45b198c1e16558a2fa2c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:32.062611 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:32.062905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:32.063186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:32.083159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:32.084406 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:32.144921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1171/2375] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:32.155676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:32.157212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:32.244044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593c9e3-bb50-4524-8a51-dc5ad956c780 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:32.248141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:32.250380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:32.250380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:32.250380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:32.259128 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:32.260832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:32.274225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1205/2376] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:08:32.695318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:32.696821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:32.712292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56620056-46a9-4b4b-aa83-55db2380488a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:32.717726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:32.718387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:32.718671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:32.718947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] PATH_INFO: `/v3/projects/9e7025a5de6c4cee87b03c401fca632d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:32.720143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7025a5de6c4cee87b03c401fca632d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:32.722060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:32.860344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7025a5de6c4cee87b03c401fca632d', 'request_id': 'req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.867333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7025a5de6c4cee87b03c401fca632d', 'request_id': 'req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.873691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7025a5de6c4cee87b03c401fca632d', 'request_id': 'req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.941757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] The token cache is being invalidate because project 9e7025a5de6c4cee87b03c401fca632d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:32.942036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.943170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:32.944536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1172/2377] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:32 2022] DELETE /identity/v3/projects/9e7025a5de6c4cee87b03c401fca632d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:32.948672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:32.953978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:32.959353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaedf9-1e1b-4939-bbc5-e1a3cb98310d service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:33.018872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:33.019672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:33.020230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:33.025364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] PATH_INFO: `/v3/users/d830544d175b40fda3b61e76482a9cd9` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:33.025364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d830544d175b40fda3b61e76482a9cd9)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:33.025364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:33.243363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd830544d175b40fda3b61e76482a9cd9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:33.301566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd830544d175b40fda3b61e76482a9cd9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:33.309578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd830544d175b40fda3b61e76482a9cd9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:33.310238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd830544d175b40fda3b61e76482a9cd9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:33.310860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-91c93c80-2ab3-4a85-8d2c-01349e4d334a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd830544d175b40fda3b61e76482a9cd9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:33.330068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1206/2378] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:32 2022] DELETE /identity/v3/users/d830544d175b40fda3b61e76482a9cd9 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:33.780956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:33.782459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:33.795804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78c7d3-b97e-469c-95b6-ec7e60bc0b60 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:33.853153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:33.853799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:33.854081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:33.854357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] PATH_INFO: `/v3/projects/55cfa24bbbf64ce9b55dfafb01e3128c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:33.855553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cfa24bbbf64ce9b55dfafb01e3128c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:33.857698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:33.941658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:33.958796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cfa24bbbf64ce9b55dfafb01e3128c', 'request_id': 'req-6579629e-415b-4da8-ab11-e24ca3adb4d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:33.964975 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cfa24bbbf64ce9b55dfafb01e3128c', 'request_id': 'req-6579629e-415b-4da8-ab11-e24ca3adb4d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:33.970771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cfa24bbbf64ce9b55dfafb01e3128c', 'request_id': 'req-6579629e-415b-4da8-ab11-e24ca3adb4d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:34.069263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] The token cache is being invalidate because project 55cfa24bbbf64ce9b55dfafb01e3128c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:34.069263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:34.069263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6579629e-415b-4da8-ab11-e24ca3adb4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:34.069263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1173/2379] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:33 2022] DELETE /identity/v3/projects/55cfa24bbbf64ce9b55dfafb01e3128c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:34.073385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ff3cb-7a84-400b-bfa0-e6c6f493e1ef None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.073879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ff3cb-7a84-400b-bfa0-e6c6f493e1ef None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.073979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ff3cb-7a84-400b-bfa0-e6c6f493e1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.094637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-995ff3cb-7a84-400b-bfa0-e6c6f493e1ef None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:34.097924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-995ff3cb-7a84-400b-bfa0-e6c6f493e1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:34.158246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1207/2380] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:34.166071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0516-5a64-4943-a509-c5f324b7554f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.166376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0516-5a64-4943-a509-c5f324b7554f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.166647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0516-5a64-4943-a509-c5f324b7554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.189745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-cf5d0516-5a64-4943-a509-c5f324b7554f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:34.191052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cf5d0516-5a64-4943-a509-c5f324b7554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:34.199200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1174/2381] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:34.203828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ff3cb-7a84-400b-bfa0-e6c6f493e1ef None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:34.213664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ff3cb-7a84-400b-bfa0-e6c6f493e1ef None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:34.218929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:34.219679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.220040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.220391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.221174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:08:34.221559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:34.223300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:34.237508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1208/2382] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:08:34.243379 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d0516-5a64-4943-a509-c5f324b7554f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:34.255447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d0516-5a64-4943-a509-c5f324b7554f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:34.260764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:34.261398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.261649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.261932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.262638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:34.264689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:34.360763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1175/2383] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:34 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:08:34.365010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:34.375667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9245c0e-57ed-4763-9c98-9238fa93e5f8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:34.380448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:34.433387 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:34.434048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.434356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.434634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.435308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:34.437349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:34.521668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1209/2384] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:34 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:34.523427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:34.534769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccf85-5481-43ec-8d55-218752922d4d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:34.539635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:34.540375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.540665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.540941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.541638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:34.543244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:34.554736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1176/2385] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:34.558816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:34.569425 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e294363-1302-454d-bd0e-13f740e7f46c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:34.574678 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77f186-662e-44b1-8c1f-979ffdc765b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:34.575513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77f186-662e-44b1-8c1f-979ffdc765b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.576155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77f186-662e-44b1-8c1f-979ffdc765b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.576435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77f186-662e-44b1-8c1f-979ffdc765b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.577242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77f186-662e-44b1-8c1f-979ffdc765b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:34.579023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77f186-662e-44b1-8c1f-979ffdc765b1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:34.593858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1210/2386] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:08:34.597543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:34.609409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d633ed-2b89-4091-a185-3df2c3a647f2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:34.614901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:34.615588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.615878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.616154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] PATH_INFO: `/v3/projects/ce44b52619484113993ada688ff41b63/users/4cb191040d5843bd94c634b82ec80884/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.632306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce44b52619484113993ada688ff41b63, user_id=4cb191040d5843bd94c634b82ec80884, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:34.635068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:34.728174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494d77b-f44b-41a2-8a37-01ab10fd80f4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.728559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494d77b-f44b-41a2-8a37-01ab10fd80f4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.728919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1177/2387] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:34 2022] PUT /identity/v3/projects/ce44b52619484113993ada688ff41b63/users/4cb191040d5843bd94c634b82ec80884/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:34.729066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494d77b-f44b-41a2-8a37-01ab10fd80f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.769443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a494d77b-f44b-41a2-8a37-01ab10fd80f4 None None] MFA Rules not processed for user `4cb191040d5843bd94c634b82ec80884`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:34.770771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a494d77b-f44b-41a2-8a37-01ab10fd80f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:34.844030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1211/2388] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:08:34.855482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:34.857632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:34.940664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bbc8f-dfdc-4f41-b358-23f3e7157ca0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:34.944426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:34.945040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:34.945322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:34.945590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:34.955108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:34.956588 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:34.965522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1178/2389] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:34.985513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494d77b-f44b-41a2-8a37-01ab10fd80f4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:35.050307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494d77b-f44b-41a2-8a37-01ab10fd80f4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:35.055672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:35.057386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:35.057386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:35.057386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] PATH_INFO: `/v3/users/4cb191040d5843bd94c634b82ec80884` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:35.058192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb191040d5843bd94c634b82ec80884)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:35.060794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:35.083771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:35.272223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb191040d5843bd94c634b82ec80884'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:35.277243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:35.279344 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:35.321945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:35.335218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb191040d5843bd94c634b82ec80884'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:35.346355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb191040d5843bd94c634b82ec80884'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:35.347802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb191040d5843bd94c634b82ec80884'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:35.349286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1de6041-fbe4-4db2-9e22-523dd3b60ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb191040d5843bd94c634b82ec80884'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:35.375080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1212/2390] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:08:34 2022] DELETE /identity/v3/users/4cb191040d5843bd94c634b82ec80884 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:35.379512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:35.379766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:35.380045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:35.393928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f04c8d-abf9-4958-8c5d-2b9cf9830478 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:35.400192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:35.401503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:35.472331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1213/2391] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:08:35.476456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:35.477375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:35.477629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:35.477971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:35.479465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:35.480120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:35.481011 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:35.494174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:35.496039 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:35.507836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1179/2392] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:35.553784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108bc22-e758-4c31-9479-8cfe3abcb917 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:35.557769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce9091-d100-494d-ba3b-0f657d723129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:35.558622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9091-d100-494d-ba3b-0f657d723129 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:35.558971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9091-d100-494d-ba3b-0f657d723129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:35.559328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9091-d100-494d-ba3b-0f657d723129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:35.579214 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce9091-d100-494d-ba3b-0f657d723129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:35.580895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce9091-d100-494d-ba3b-0f657d723129 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:35.599727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1214/2393] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:36.022174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:36.023785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:36.050908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd4b0-b33d-4c7a-a79a-9a53dc7165ab service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:36.074570 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:36.076599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.076945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.077225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] PATH_INFO: `/v3/projects/ce44b52619484113993ada688ff41b63` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.078651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce44b52619484113993ada688ff41b63)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:36.080704 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:36.235971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce44b52619484113993ada688ff41b63', 'request_id': 'req-2e0065f8-352c-4352-be86-aa8e78ea4470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:36.243288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce44b52619484113993ada688ff41b63', 'request_id': 'req-2e0065f8-352c-4352-be86-aa8e78ea4470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:36.251262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce44b52619484113993ada688ff41b63', 'request_id': 'req-2e0065f8-352c-4352-be86-aa8e78ea4470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:36.353147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] The token cache is being invalidate because project ce44b52619484113993ada688ff41b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:08:36.353444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:36.356858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2e0065f8-352c-4352-be86-aa8e78ea4470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:08:36.356858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1180/2394] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:08:36 2022] DELETE /identity/v3/projects/ce44b52619484113993ada688ff41b63 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:08:36.361452 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cc4b3-4d9d-4c8e-ba97-7e37a624efb0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.361962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cc4b3-4d9d-4c8e-ba97-7e37a624efb0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.362075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cc4b3-4d9d-4c8e-ba97-7e37a624efb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.384631 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-316cc4b3-4d9d-4c8e-ba97-7e37a624efb0 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:36.386670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-316cc4b3-4d9d-4c8e-ba97-7e37a624efb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:36.426130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:36.459183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1215/2395] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:36.468009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064489f0-08cf-4def-86a4-9806e1755405 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.468361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064489f0-08cf-4def-86a4-9806e1755405 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.468711 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064489f0-08cf-4def-86a4-9806e1755405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.491392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-064489f0-08cf-4def-86a4-9806e1755405 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:36.492795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-064489f0-08cf-4def-86a4-9806e1755405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:36.502636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1181/2396] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:36.507087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cc4b3-4d9d-4c8e-ba97-7e37a624efb0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:36.518512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cc4b3-4d9d-4c8e-ba97-7e37a624efb0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:36.524823 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:36.525476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.525765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.526045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.526771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:08:36.527086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:36.529060 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:36.544390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1216/2397] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:08:36.556362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064489f0-08cf-4def-86a4-9806e1755405 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:36.567311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064489f0-08cf-4def-86a4-9806e1755405 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:36.572433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:36.575957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.575957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.575957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.575957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:36.576867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:36.582175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:36.648411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1182/2398] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:08:36 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:08:36.652459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:36.664908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3674b4-55c3-4736-afbd-decdbc3358ef admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:36.727647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:36.728619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.728913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.729191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.729865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:36.732464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:36.851043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1217/2399] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:08:36 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:08:36.855045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:36.866595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade4a8c-edd7-4260-a084-5ca797f8989e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:36.871171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:36.871992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.872205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.872477 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.873170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:36.874769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:36.885216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1183/2400] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:36.891797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:36.903780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bc02a-6265-479e-b156-5dfa317c396f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:36.909270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d5168-a222-4df0-86d2-d46eb395130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:36.910226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d5168-a222-4df0-86d2-d46eb395130a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.910515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d5168-a222-4df0-86d2-d46eb395130a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.910792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d5168-a222-4df0-86d2-d46eb395130a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.911490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d5168-a222-4df0-86d2-d46eb395130a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:36.913122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d5168-a222-4df0-86d2-d46eb395130a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:36.925942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1218/2401] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:08:36.930135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:36.940307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807d343-e557-4dc7-a8e8-66cc27b39d3a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:36.944893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:36.945564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:36.945853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:36.946132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] PATH_INFO: `/v3/projects/375a8e5d4b6f457ea7ab404a47177662/users/5afb3694f78849a6a13e78958e6fa56d/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:36.961864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375a8e5d4b6f457ea7ab404a47177662, user_id=5afb3694f78849a6a13e78958e6fa56d, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:36.964612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:37.056095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1184/2402] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:08:36 2022] PUT /identity/v3/projects/375a8e5d4b6f457ea7ab404a47177662/users/5afb3694f78849a6a13e78958e6fa56d/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:08:37.066647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d817b3d-1576-4ec8-8e47-8d2c4255a7b2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:37.066942 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d817b3d-1576-4ec8-8e47-8d2c4255a7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:37.067222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d817b3d-1576-4ec8-8e47-8d2c4255a7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:37.097158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4d817b3d-1576-4ec8-8e47-8d2c4255a7b2 None None] MFA Rules not processed for user `5afb3694f78849a6a13e78958e6fa56d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:08:37.098376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4d817b3d-1576-4ec8-8e47-8d2c4255a7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:37.166858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1219/2403] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:08:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:08:37.179207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:08:37.184739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:08:37.244152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ebe8-3065-4c4e-b71e-e88330d4bc32 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:08:37.247929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340b625-61b4-4c5c-9f40-f4ec121d7f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:08:37.248562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340b625-61b4-4c5c-9f40-f4ec121d7f71 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:08:37.248879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340b625-61b4-4c5c-9f40-f4ec121d7f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:08:37.249160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340b625-61b4-4c5c-9f40-f4ec121d7f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:08:37.258173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340b625-61b4-4c5c-9f40-f4ec121d7f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:08:37.259652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340b625-61b4-4c5c-9f40-f4ec121d7f71 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:08:37.269205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1185/2404] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:08:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:08:47.163801 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:08:47.265125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:05.332239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d817b3d-1576-4ec8-8e47-8d2c4255a7b2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:05.393994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d817b3d-1576-4ec8-8e47-8d2c4255a7b2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:05.398658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:05.399295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:05.399607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:05.399889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] PATH_INFO: `/v3/users/5afb3694f78849a6a13e78958e6fa56d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:05.401044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5afb3694f78849a6a13e78958e6fa56d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:05.402988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:05.555548 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5afb3694f78849a6a13e78958e6fa56d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:05.626489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5afb3694f78849a6a13e78958e6fa56d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:05.633963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5afb3694f78849a6a13e78958e6fa56d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:05.634584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5afb3694f78849a6a13e78958e6fa56d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:05.635183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5afb3694f78849a6a13e78958e6fa56d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:05.653469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1220/2405] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:09:05 2022] DELETE /identity/v3/users/5afb3694f78849a6a13e78958e6fa56d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:09:05.658717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:05.659230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:05.659337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:05.680181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:05.681901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:05.743124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1186/2406] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:09:05.758143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:05.759615 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:05.826180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a8501-3835-4446-a2f0-07e3532ebf53 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:05.830549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b9818-cd99-4433-be8a-3960d1ef7b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:05.831180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9818-cd99-4433-be8a-3960d1ef7b53 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:05.831470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9818-cd99-4433-be8a-3960d1ef7b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:05.831785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9818-cd99-4433-be8a-3960d1ef7b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:05.841364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9818-cd99-4433-be8a-3960d1ef7b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:05.842828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9818-cd99-4433-be8a-3960d1ef7b53 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:05.853617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1221/2407] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:09:06.263304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:06.269302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:06.282196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646fd0-8e1e-4baa-87b4-89f8dc722bdf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:06.287642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:06.288584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:06.288736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:06.289078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] PATH_INFO: `/v3/projects/375a8e5d4b6f457ea7ab404a47177662` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:06.290273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=375a8e5d4b6f457ea7ab404a47177662)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:06.292364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:06.405035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375a8e5d4b6f457ea7ab404a47177662', 'request_id': 'req-962b2c24-ab27-44fa-abfe-7010080110ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:06.412112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375a8e5d4b6f457ea7ab404a47177662', 'request_id': 'req-962b2c24-ab27-44fa-abfe-7010080110ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:06.418419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375a8e5d4b6f457ea7ab404a47177662', 'request_id': 'req-962b2c24-ab27-44fa-abfe-7010080110ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:06.501396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] The token cache is being invalidate because project 375a8e5d4b6f457ea7ab404a47177662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:09:06.501691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:06.502332 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-962b2c24-ab27-44fa-abfe-7010080110ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:09:06.503956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1187/2408] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:09:06 2022] DELETE /identity/v3/projects/375a8e5d4b6f457ea7ab404a47177662 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:09:06.509040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eff4dc-f5ef-49cb-9163-68fad330a347 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:06.509498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eff4dc-f5ef-49cb-9163-68fad330a347 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:06.509607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eff4dc-f5ef-49cb-9163-68fad330a347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:06.530267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-34eff4dc-f5ef-49cb-9163-68fad330a347 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:06.531608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-34eff4dc-f5ef-49cb-9163-68fad330a347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:06.548937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:06.592537 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1222/2409] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:09:06.602078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebc8b-f372-4384-a367-b6db8bb606bc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:06.602402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebc8b-f372-4384-a367-b6db8bb606bc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:06.602641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebc8b-f372-4384-a367-b6db8bb606bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:06.626609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f86ebc8b-f372-4384-a367-b6db8bb606bc None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:06.628589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f86ebc8b-f372-4384-a367-b6db8bb606bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:06.641032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1188/2410] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:09:06.643026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eff4dc-f5ef-49cb-9163-68fad330a347 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:06.653778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eff4dc-f5ef-49cb-9163-68fad330a347 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:06.659017 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:06.659690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:06.659977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:06.660256 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:06.660966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:09:06.661562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:06.663610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:06.676259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1223/2411] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:09:06.680400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ebc8b-f372-4384-a367-b6db8bb606bc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:06.690940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ebc8b-f372-4384-a367-b6db8bb606bc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:06.696597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:06.697309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:06.697669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:06.698038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:06.698781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:06.700814 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:06.827416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1189/2412] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:09:06 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:09:06.829419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:06.840725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71119f-17c9-4a46-ae30-fd4301b33a17 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:06.900729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:06.901378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:06.901742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:06.902026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:06.902728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:06.904709 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:06.992849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1224/2413] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:09:06 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:09:06.997150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:07.008655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fce7d8-e167-406a-8a2d-3db2924a12ae admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:07.016447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:07.016447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:07.016447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:07.016447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:07.016869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:07.017113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:07.027240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1190/2414] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:09:07.031476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:07.043826 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594515e-519f-466f-8a40-a2eb030f9f49 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:07.049609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02483a0-1f36-4ecc-a09c-ea01a70bf642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:07.050229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02483a0-1f36-4ecc-a09c-ea01a70bf642 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:07.050511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02483a0-1f36-4ecc-a09c-ea01a70bf642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:07.050786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02483a0-1f36-4ecc-a09c-ea01a70bf642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:07.051518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02483a0-1f36-4ecc-a09c-ea01a70bf642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:07.053173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02483a0-1f36-4ecc-a09c-ea01a70bf642 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:07.072432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1225/2415] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:09:07.073566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:07.086400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d357d-d014-4b8a-a798-d1b33a188316 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:07.092388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:07.097424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:07.097424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:07.097424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] PATH_INFO: `/v3/projects/eb938950a17942b9b4b1ca56f319e754/users/1bc01a0e20fd411b9df5d2f077784f33/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:07.110099 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb938950a17942b9b4b1ca56f319e754, user_id=1bc01a0e20fd411b9df5d2f077784f33, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:07.112820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:07.219133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1191/2416] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:09:07 2022] PUT /identity/v3/projects/eb938950a17942b9b4b1ca56f319e754/users/1bc01a0e20fd411b9df5d2f077784f33/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:09:07.224522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:07.224817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:07.225096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:07.244879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:07.246085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:07.313570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1226/2417] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:09:07.324731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:07.326597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:07.363386 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:07.421069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdad5c6-21fc-4060-b528-556b90853a1f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:07.426065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a500c-9d77-4d34-8cd8-242b8dab679d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:07.426723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a500c-9d77-4d34-8cd8-242b8dab679d service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:07.427003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a500c-9d77-4d34-8cd8-242b8dab679d service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:07.427282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a500c-9d77-4d34-8cd8-242b8dab679d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:07.438363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a500c-9d77-4d34-8cd8-242b8dab679d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:07.440061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a500c-9d77-4d34-8cd8-242b8dab679d service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:07.450839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1192/2418] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:09:07.770957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:07.776908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:07.817215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:07.889805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced7d37-ddde-4744-a366-c39fe51bfc66 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:07.895724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb70e92-f0f9-479a-88d6-20704150036c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:07.896497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70e92-f0f9-479a-88d6-20704150036c service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:07.896848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70e92-f0f9-479a-88d6-20704150036c service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:07.897195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70e92-f0f9-479a-88d6-20704150036c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:07.900090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cbb70e92-f0f9-479a-88d6-20704150036c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:07.915550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb70e92-f0f9-479a-88d6-20704150036c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:07.917857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb70e92-f0f9-479a-88d6-20704150036c service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:07.931948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1227/2419] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:09:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:09:12.638492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51c7ed-0b27-4286-86bf-4f5fd4c704e4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:12.639023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51c7ed-0b27-4286-86bf-4f5fd4c704e4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:12.639413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51c7ed-0b27-4286-86bf-4f5fd4c704e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:12.661092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9f51c7ed-0b27-4286-86bf-4f5fd4c704e4 None None] MFA Rules not processed for user `0d2c7d95047445ee9d44640b5372e0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:12.662411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9f51c7ed-0b27-4286-86bf-4f5fd4c704e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:12.731043 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1193/2420] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:09:13.380056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6dfde-8fb4-47d3-8a2f-8547150e3ab4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:13.380959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6dfde-8fb4-47d3-8a2f-8547150e3ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:13.381245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6dfde-8fb4-47d3-8a2f-8547150e3ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:13.414153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9ca6dfde-8fb4-47d3-8a2f-8547150e3ab4 None None] MFA Rules not processed for user `1bc01a0e20fd411b9df5d2f077784f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:13.415305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9ca6dfde-8fb4-47d3-8a2f-8547150e3ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:13.501208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1228/2421] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:09:13.505583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51c7ed-0b27-4286-86bf-4f5fd4c704e4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:13.518432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51c7ed-0b27-4286-86bf-4f5fd4c704e4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:13.524179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:13.525079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:13.525501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:13.525844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:13.526595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:13.528890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:13.656502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1194/2422] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:09:13 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:09:13.660261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6dfde-8fb4-47d3-8a2f-8547150e3ab4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:13.672186 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6dfde-8fb4-47d3-8a2f-8547150e3ab4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:13.743563 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:13.744582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:13.744582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:13.745218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:13.745595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:13.747772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:13.863309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1229/2423] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:09:13 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:09:13.870091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:13.882411 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b0a2-d607-4a24-a1b1-6597f6afbb82 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:13.887444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:13.888295 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:13.888736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:13.889162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:13.890567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:13.892471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:13.906682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1195/2424] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:09:13.910706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:13.921243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb4d59-be74-4293-b082-4d359bf6be46 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:13.926886 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:13.931612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:13.931612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:13.931612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] PATH_INFO: `/v3/projects/b7213e8868bc4d9086c873a6fe0d49ab/users/aa8f7ec2f17b407faa2180ea8070549f/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:13.946553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7213e8868bc4d9086c873a6fe0d49ab, user_id=aa8f7ec2f17b407faa2180ea8070549f, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:13.949357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:13.952206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:14.039471 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1230/2425] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:09:13 2022] PUT /identity/v3/projects/b7213e8868bc4d9086c873a6fe0d49ab/users/aa8f7ec2f17b407faa2180ea8070549f/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:09:14.043654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:14.055239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adb741-02ad-48e4-afd7-1b30d2c12f91 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:14.120837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab84385-f0be-4311-a339-230634019afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:14.121468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab84385-f0be-4311-a339-230634019afe admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:14.121753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab84385-f0be-4311-a339-230634019afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:14.122028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab84385-f0be-4311-a339-230634019afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:14.122724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab84385-f0be-4311-a339-230634019afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:14.124514 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab84385-f0be-4311-a339-230634019afe admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:14.140746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1196/2426] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:09:14.140869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:14.152045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed59b3-3c39-4fc2-a8a9-b455fc2f0108 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:14.157795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:14.158486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:14.158775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:14.159221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] PATH_INFO: `/v3/projects/b7213e8868bc4d9086c873a6fe0d49ab/users/aa8f7ec2f17b407faa2180ea8070549f/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:14.161295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7213e8868bc4d9086c873a6fe0d49ab, user_id=aa8f7ec2f17b407faa2180ea8070549f, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:14.164107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:14.183614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1231/2427] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:09:14 2022] PUT /identity/v3/projects/b7213e8868bc4d9086c873a6fe0d49ab/users/aa8f7ec2f17b407faa2180ea8070549f/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:09:20.654839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900331ab-c127-465e-813c-1f61d729f3a8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:20.655993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900331ab-c127-465e-813c-1f61d729f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:20.656463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900331ab-c127-465e-813c-1f61d729f3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:20.691161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-900331ab-c127-465e-813c-1f61d729f3a8 None None] MFA Rules not processed for user `aa8f7ec2f17b407faa2180ea8070549f`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:20.692559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-900331ab-c127-465e-813c-1f61d729f3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:20.717301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:20.777728 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1197/2428] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:09:20.791066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:20.792917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:20.874913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fb865-a62f-47a5-96fc-82199dc055ff admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:20.880582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:20.881227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:20.881516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:20.881827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:20.941943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:20.943478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:20.953149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1232/2429] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:09:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:09:23.498033 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8142e-fbf5-4847-9359-aa6733bde840 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:23.498540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8142e-fbf5-4847-9359-aa6733bde840 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:23.498635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8142e-fbf5-4847-9359-aa6733bde840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:23.518550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-15d8142e-fbf5-4847-9359-aa6733bde840 None None] MFA Rules not processed for user `1bc01a0e20fd411b9df5d2f077784f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:09:23.519897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-15d8142e-fbf5-4847-9359-aa6733bde840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:23.527456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1198/2430] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:09:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:09:30.948666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:INSERT=2,keystone:SELECT=63 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:33.513410 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=23 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:09:52.126145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:09:52.128207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:52.191084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b87c8-1f52-4733-96af-0d6b64720f16 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:09:52.194995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:09:52.195650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:09:52.195937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:09:52.196593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:09:52.197972 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:09:52.263089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:09:52.264704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:09:52.274041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1233/2431] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:09:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:10:02.269717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=48 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:10:04.880439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8ed33-2122-46fd-9a68-253f09b62dd2 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:04.880968 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8ed33-2122-46fd-9a68-253f09b62dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:04.881068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8ed33-2122-46fd-9a68-253f09b62dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:04.901329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-12e8ed33-2122-46fd-9a68-253f09b62dd2 None None] MFA Rules not processed for user `d4d85df24c9e4312bbe2da197f55e116`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:10:04.902547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-12e8ed33-2122-46fd-9a68-253f09b62dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:04.909476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1199/2432] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:10:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:10:13.356003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:10:13.410660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75450a-adf3-4a30-b5bd-7523ebbf3f5e service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:10:13.414319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:10:13.414932 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:13.415267 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:13.415750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:13.425029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:10:13.426476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:10:13.437375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1234/2433] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:10:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:10:14.895399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:10:23.433292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=26 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:10:27.418557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8ed33-2122-46fd-9a68-253f09b62dd2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:10:27.428456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8ed33-2122-46fd-9a68-253f09b62dd2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:10:27.432205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:10:27.432817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:27.433105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:27.433385 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:27.435167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:27.498389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:10:27.499886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:10:27.509811 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1200/2434] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:10:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:10:32.573700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:10:32.582920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c724df6-0718-48fd-b46c-16add6fe69c4 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:10:32.586499 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92467628-0e78-4ca1-b911-04d63dafaf6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:10:32.587101 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92467628-0e78-4ca1-b911-04d63dafaf6a service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:32.587385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92467628-0e78-4ca1-b911-04d63dafaf6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:32.588321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92467628-0e78-4ca1-b911-04d63dafaf6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:32.590228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92467628-0e78-4ca1-b911-04d63dafaf6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:32.603239 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92467628-0e78-4ca1-b911-04d63dafaf6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:10:32.605355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92467628-0e78-4ca1-b911-04d63dafaf6a service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:10:32.616194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1235/2435] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:10:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:10:36.958683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:10:36.968387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251dbc5-5488-4c93-ad7b-e168b296970b service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:10:36.972080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:10:36.972683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:36.972963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:36.973233 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:36.974986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:36.989619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:10:36.991058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:10:37.002302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1201/2436] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:10:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:10:42.611352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:10:46.997407 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=36 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:10:56.977641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:56.978122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:56.982269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:57.040199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:10:57.041437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:57.112578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1236/2437] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:10:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:10:57.241580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:10:57.243013 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:57.335015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04441d75-e6e2-4350-bd2a-a99eeaefaf48 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:10:57.340225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:10:57.341462 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:57.341687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:57.342080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:57.350701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:10:57.352325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:10:57.362842 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1202/2438] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:10:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 08:10:57.422998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:10:57.424579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:57.440840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f9bb3-51d3-419d-8f3f-1551cf57d626 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:10:57.445211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:10:57.445835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:10:57.446143 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:10:57.446420 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:10:57.448303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:10:57.526796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:10:57.529672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:10:57.541140 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1237/2439] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:01.871518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:02.004700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5a9b-b156-43c4-8644-470819597e84 service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:02.012165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:02.013746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:02.014229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:02.014503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:02.016479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:02.032851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:02.036845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:02.063922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fae87-00dc-4a63-a5e2-12c11b935aa1 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:02.064435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:02.064435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:02.071833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:02.074766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:02.074766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:02.074766 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] PATH_INFO: `/v3/users/f030155981d741f1a42b9b84ca5d6af4` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:02.075593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1203/2440] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:02.091779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f030155981d741f1a42b9b84ca5d6af4)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:02.100254 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:02.281240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:02.287871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f030155981d741f1a42b9b84ca5d6af4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.363961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f030155981d741f1a42b9b84ca5d6af4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.374769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f030155981d741f1a42b9b84ca5d6af4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.380514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f030155981d741f1a42b9b84ca5d6af4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.380514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d90e52d-68b4-412c-a6d6-fd498fc20ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f030155981d741f1a42b9b84ca5d6af4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.397341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1238/2441] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:11:02 2022] DELETE /identity/v3/users/f030155981d741f1a42b9b84ca5d6af4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:11:02.655408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:02.690838 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0afbc8-d11c-49f5-a56e-2122d1481cec service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:02.777177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:02.777942 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:02.778404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:02.778821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] PATH_INFO: `/v3/projects/606b2e9ca91341fe9bbe157935c5736e` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:02.781639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:02.791892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=606b2e9ca91341fe9bbe157935c5736e)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:02.794623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:02.888334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606b2e9ca91341fe9bbe157935c5736e', 'request_id': 'req-22fa258c-a624-4804-973d-b75cb0378e76'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.899152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606b2e9ca91341fe9bbe157935c5736e', 'request_id': 'req-22fa258c-a624-4804-973d-b75cb0378e76'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.905784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606b2e9ca91341fe9bbe157935c5736e', 'request_id': 'req-22fa258c-a624-4804-973d-b75cb0378e76'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.957061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] The token cache is being invalidate because project 606b2e9ca91341fe9bbe157935c5736e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:11:02.957455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.957999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-22fa258c-a624-4804-973d-b75cb0378e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:02.961397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1204/2442] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:11:02 2022] DELETE /identity/v3/projects/606b2e9ca91341fe9bbe157935c5736e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:02.967427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081dd54-842b-41ed-8526-40c7853c3b11 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:02.967757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081dd54-842b-41ed-8526-40c7853c3b11 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:02.968049 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081dd54-842b-41ed-8526-40c7853c3b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:02.990286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c081dd54-842b-41ed-8526-40c7853c3b11 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:02.991557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c081dd54-842b-41ed-8526-40c7853c3b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:03.061965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1239/2443] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:11:03.070875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9ebaf-d6fd-4f2a-8744-57e3720b1f29 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.072823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9ebaf-d6fd-4f2a-8744-57e3720b1f29 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.072823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9ebaf-d6fd-4f2a-8744-57e3720b1f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.102645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-25e9ebaf-d6fd-4f2a-8744-57e3720b1f29 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:03.104059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-25e9ebaf-d6fd-4f2a-8744-57e3720b1f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:03.114630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1205/2444] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:03.118805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081dd54-842b-41ed-8526-40c7853c3b11 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:03.130450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081dd54-842b-41ed-8526-40c7853c3b11 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:03.136050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:03.137144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.137144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.137144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.137955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:11:03.138334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:03.140279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:03.156170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1240/2445] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:11:03.160555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9ebaf-d6fd-4f2a-8744-57e3720b1f29 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:03.174815 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9ebaf-d6fd-4f2a-8744-57e3720b1f29 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:03.181019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:03.181656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.181945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.182226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.182898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:03.185339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:03.292692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1206/2446] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:11:03 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:03.294561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:03.306024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76de0b5-78fc-4bc1-a3d4-008a00b1e6f3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:03.365173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:03.365924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.366260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.366602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.367373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:03.369430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:03.435432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1241/2447] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:11:03 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:03.436648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:03.447986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a5c50-8d5c-4103-b3f2-b026c2459448 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:03.452851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:03.453592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.453985 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.454555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.455413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:03.457131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:03.468664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1207/2448] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:03.476246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:03.488030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad895-b59b-4b91-9d71-2e253190fd01 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:03.494423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504033b-c2fe-4be2-aae4-7a8c2d13041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:03.495356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504033b-c2fe-4be2-aae4-7a8c2d13041e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.495677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504033b-c2fe-4be2-aae4-7a8c2d13041e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.495961 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504033b-c2fe-4be2-aae4-7a8c2d13041e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.496730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504033b-c2fe-4be2-aae4-7a8c2d13041e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:03.498524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504033b-c2fe-4be2-aae4-7a8c2d13041e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:03.511566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1242/2449] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:03.514965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:03.525137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1582f-646c-40e2-a44f-4d873efe1e6d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:03.537544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:03.538308 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.538667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.539028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] PATH_INFO: `/v3/projects/fd818fdc14a84f30816843e3b7279f77/users/9710c7e0f16848408b6b17e7c073af56/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.561178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd818fdc14a84f30816843e3b7279f77, user_id=9710c7e0f16848408b6b17e7c073af56, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:03.565747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:03.638549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1208/2450] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:11:03 2022] PUT /identity/v3/projects/fd818fdc14a84f30816843e3b7279f77/users/9710c7e0f16848408b6b17e7c073af56/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:11:03.649086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb1d4f-7434-421c-8b59-214d5e1e5430 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.649417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb1d4f-7434-421c-8b59-214d5e1e5430 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.649849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb1d4f-7434-421c-8b59-214d5e1e5430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.681089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-90eb1d4f-7434-421c-8b59-214d5e1e5430 None None] MFA Rules not processed for user `9710c7e0f16848408b6b17e7c073af56`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:03.682807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-90eb1d4f-7434-421c-8b59-214d5e1e5430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:03.705348 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:03.778523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1243/2451] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:03.789625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:03.791510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:03.886435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e96ed5-4b2a-45aa-a9f9-7f7bf66e79b9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:03.893030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4575cf-ff09-45bd-ba4e-70278f702932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:03.894328 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4575cf-ff09-45bd-ba4e-70278f702932 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:03.894784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4575cf-ff09-45bd-ba4e-70278f702932 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:03.895060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4575cf-ff09-45bd-ba4e-70278f702932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:03.900288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:03.904852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4575cf-ff09-45bd-ba4e-70278f702932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:03.906356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4575cf-ff09-45bd-ba4e-70278f702932 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:03.915961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1209/2452] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:08.681501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb1d4f-7434-421c-8b59-214d5e1e5430 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:08.750744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb1d4f-7434-421c-8b59-214d5e1e5430 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:08.756058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:08.760541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:08.760541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:08.760541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] PATH_INFO: `/v3/users/9710c7e0f16848408b6b17e7c073af56` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:08.760541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9710c7e0f16848408b6b17e7c073af56)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:08.760541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:08.980963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9710c7e0f16848408b6b17e7c073af56'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:09.060572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9710c7e0f16848408b6b17e7c073af56'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:09.092193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9710c7e0f16848408b6b17e7c073af56'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:09.093147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9710c7e0f16848408b6b17e7c073af56'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:09.093778 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9710c7e0f16848408b6b17e7c073af56'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:09.138736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1244/2453] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:11:08 2022] DELETE /identity/v3/users/9710c7e0f16848408b6b17e7c073af56 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:09.144010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:09.144963 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:09.145455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:09.177305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:09.178605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:09.277424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1210/2454] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:09.282955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:09.284449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:09.329612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:09.353874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdde5a-5931-415d-96fd-2fa663ed603b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:09.357997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b025b0a-f24e-4d80-a7ab-256d446b0e9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:09.358754 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b025b0a-f24e-4d80-a7ab-256d446b0e9c service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:09.359110 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b025b0a-f24e-4d80-a7ab-256d446b0e9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:09.359464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b025b0a-f24e-4d80-a7ab-256d446b0e9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:09.376568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b025b0a-f24e-4d80-a7ab-256d446b0e9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:09.378796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b025b0a-f24e-4d80-a7ab-256d446b0e9c service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:09.391612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1245/2455] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:09.895004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:09.895004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:09.913683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ca01c-39ac-4d9a-8752-21c0c589775e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:09.919464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:09.920194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:09.920589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:09.920739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] PATH_INFO: `/v3/projects/fd818fdc14a84f30816843e3b7279f77` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:09.931404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd818fdc14a84f30816843e3b7279f77)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:09.933494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.022720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd818fdc14a84f30816843e3b7279f77', 'request_id': 'req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:10.029885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd818fdc14a84f30816843e3b7279f77', 'request_id': 'req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:10.038805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd818fdc14a84f30816843e3b7279f77', 'request_id': 'req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:10.087795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] The token cache is being invalidate because project fd818fdc14a84f30816843e3b7279f77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:11:10.088087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:10.088713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7cfc3758-ccdb-4012-80d5-f8d2a6d33380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:10.090378 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1211/2456] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:11:09 2022] DELETE /identity/v3/projects/fd818fdc14a84f30816843e3b7279f77 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:10.096264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620699b3-c913-4577-bcf0-8b9826ca4853 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.096768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620699b3-c913-4577-bcf0-8b9826ca4853 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.096864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620699b3-c913-4577-bcf0-8b9826ca4853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.121428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-620699b3-c913-4577-bcf0-8b9826ca4853 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:10.122658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-620699b3-c913-4577-bcf0-8b9826ca4853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:10.189486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1246/2457] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:10.198034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2b420-22b4-44a6-8796-a85cd9a96ba3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.198550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2b420-22b4-44a6-8796-a85cd9a96ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.198660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2b420-22b4-44a6-8796-a85cd9a96ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.221589 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-91d2b420-22b4-44a6-8796-a85cd9a96ba3 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:10.222828 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-91d2b420-22b4-44a6-8796-a85cd9a96ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:10.231432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1212/2458] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:10.235936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620699b3-c913-4577-bcf0-8b9826ca4853 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.246347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620699b3-c913-4577-bcf0-8b9826ca4853 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.252114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.252817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.253098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.253372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.254490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:11:10.254807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.256503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.273677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1247/2459] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:11:10.279381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2b420-22b4-44a6-8796-a85cd9a96ba3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.296777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2b420-22b4-44a6-8796-a85cd9a96ba3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.302204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.302833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.303118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.303393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.304096 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.305998 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.394048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1213/2460] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:11:10 2022] POST /identity/v3/projects => generated 405 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:10.398077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.408617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c971af-ba1b-4e70-be98-a221df41a553 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.477829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.478462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.478746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.479022 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.479727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.481887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.574502 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1248/2461] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:11:10 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:10.583753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.594508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e557ebe-b16f-4a14-908d-b63cd2ac8cd5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.599294 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.600012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.600297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.600580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.601310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.602995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.614363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1214/2462] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:10.618472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.629386 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb812b5-82cc-49de-a988-812352645613 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.634054 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ddf9d-fc19-4674-abb0-eb395d7b5e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.634665 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ddf9d-fc19-4674-abb0-eb395d7b5e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.634949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ddf9d-fc19-4674-abb0-eb395d7b5e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.635227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ddf9d-fc19-4674-abb0-eb395d7b5e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.635964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ddf9d-fc19-4674-abb0-eb395d7b5e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.637588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ddf9d-fc19-4674-abb0-eb395d7b5e79 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.648897 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1249/2463] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:10.653183 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.663939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a29c15-cf84-4a82-b76b-56f00260ad86 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.668548 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.669206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.669491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.669769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] PATH_INFO: `/v3/projects/b539bf6ef01e43c3887a07e539e66d99/users/7159bf77fcaa445e96b7e52f97e447b8/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.684656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b539bf6ef01e43c3887a07e539e66d99, user_id=7159bf77fcaa445e96b7e52f97e447b8, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.687375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.786490 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1215/2464] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:11:10 2022] PUT /identity/v3/projects/b539bf6ef01e43c3887a07e539e66d99/users/7159bf77fcaa445e96b7e52f97e447b8/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:10.798304 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.798592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.798870 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.828489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] MFA Rules not processed for user `7159bf77fcaa445e96b7e52f97e447b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:10.830303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:10.860130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:10.895206 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1250/2465] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:10.900009 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.913711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:10.915185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:10.916299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:10.967435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69708dc-8680-4cb2-a805-02ca959bfc9e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.972644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.973397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.973691 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.973972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.974146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf33f4-3152-4dc6-92e0-4b50b533b2e2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:10.974705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.976601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:10.978401 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:10.979014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:10.979296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:10.979636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:10.981406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:10.991036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:10.992508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:11.000515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1251/2466] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:11:11.090290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1216/2467] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:11:10 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:11:11.094238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:11.106055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baa6cf-2cbc-4f5b-9924-a6b091614e81 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:11.165170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:11.165803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:11.166094 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:11.166409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:11.167079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:11.169050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:11.288733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1252/2468] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:11:11 2022] POST /identity/v3/users => generated 309 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:11.292819 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:11.304502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5790a76-5c7d-4350-a94d-d45b45012e7f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:11.309512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:11.310141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:11.310521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:11.310805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:11.311508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:11.313192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:11.325171 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1217/2469] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:11.329531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:11.340008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68298c-4ce5-4802-815e-387f9632bc25 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:11.344821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:11.345874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:11.346167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:11.346451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] PATH_INFO: `/v3/projects/79eccf08a6514edcb30424c77f0ccb38/users/ba7e7d2c8e3f4addb9b069e33f5921a8/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:11.362596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79eccf08a6514edcb30424c77f0ccb38, user_id=ba7e7d2c8e3f4addb9b069e33f5921a8, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:11.365479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:11.459750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1253/2470] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:11:11 2022] PUT /identity/v3/projects/79eccf08a6514edcb30424c77f0ccb38/users/ba7e7d2c8e3f4addb9b069e33f5921a8/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:11.463771 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:11.474162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c2b3-94f6-4290-b319-457519abe506 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:11.553120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060ae90-a62e-442b-8e32-b7417a99edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:11.553757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ae90-a62e-442b-8e32-b7417a99edfa admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:11.554041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ae90-a62e-442b-8e32-b7417a99edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:11.554317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ae90-a62e-442b-8e32-b7417a99edfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:11.555101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060ae90-a62e-442b-8e32-b7417a99edfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:11.556748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060ae90-a62e-442b-8e32-b7417a99edfa admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:11.568449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1218/2471] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:11.573818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:11.583053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255287ec-d186-4a1c-aee0-805ceb8f0229 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:11.587887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:11.588562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:11.588847 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:11.589125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] PATH_INFO: `/v3/projects/79eccf08a6514edcb30424c77f0ccb38/users/ba7e7d2c8e3f4addb9b069e33f5921a8/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:11.591106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79eccf08a6514edcb30424c77f0ccb38, user_id=ba7e7d2c8e3f4addb9b069e33f5921a8, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:11.593863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:11.605037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1254/2472] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:11:11 2022] PUT /identity/v3/projects/79eccf08a6514edcb30424c77f0ccb38/users/ba7e7d2c8e3f4addb9b069e33f5921a8/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:11:11.615929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb66b-d786-4e92-a72e-deb141d18c28 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:11.619697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb66b-d786-4e92-a72e-deb141d18c28 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:11.619697 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb66b-d786-4e92-a72e-deb141d18c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:11.646647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-cb7fb66b-d786-4e92-a72e-deb141d18c28 None None] MFA Rules not processed for user `ba7e7d2c8e3f4addb9b069e33f5921a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:11.647932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cb7fb66b-d786-4e92-a72e-deb141d18c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:11.722193 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1219/2473] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:11.732833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:11.734547 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:11.766108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:11.793026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83500f76-3311-4db2-b63a-9cbd93093985 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:11.797005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7069a-83dd-42c1-b710-f6ee394e3f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:11.797632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7069a-83dd-42c1-b710-f6ee394e3f6a service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:11.797918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7069a-83dd-42c1-b710-f6ee394e3f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:11.798268 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7069a-83dd-42c1-b710-f6ee394e3f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:11.808280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7069a-83dd-42c1-b710-f6ee394e3f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:11.809769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7069a-83dd-42c1-b710-f6ee394e3f6a service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:11.819517 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1255/2474] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:11 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:11.956777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fb66b-d786-4e92-a72e-deb141d18c28 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:11.966541 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:12.020182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fb66b-d786-4e92-a72e-deb141d18c28 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:12.025027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:12.025674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:12.025962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:12.026241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] PATH_INFO: `/v3/users/7159bf77fcaa445e96b7e52f97e447b8` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:12.027403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7159bf77fcaa445e96b7e52f97e447b8)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:12.029365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:12.267864 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7159bf77fcaa445e96b7e52f97e447b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:12.275103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7159bf77fcaa445e96b7e52f97e447b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:12.333855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7159bf77fcaa445e96b7e52f97e447b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:12.334496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7159bf77fcaa445e96b7e52f97e447b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:12.346772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7159bf77fcaa445e96b7e52f97e447b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:12.352710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1220/2475] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:11:11 2022] DELETE /identity/v3/users/7159bf77fcaa445e96b7e52f97e447b8 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:12.357314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:12.357768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:12.357883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:12.378236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:12.380055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:12.446731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1256/2476] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:12.456507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:12.457924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:12.514898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaa5a8-e79c-4b83-bd89-61478085151b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:12.518576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:12.519248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:12.519564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:12.519853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:12.528823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:12.530297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:12.540254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1221/2477] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:12.941882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:12.943398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:12.955779 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaec46-9fdf-47a8-a09f-6a6b2a0bda6b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:12.960734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:12.961556 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:12.961844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:12.962127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] PATH_INFO: `/v3/projects/b539bf6ef01e43c3887a07e539e66d99` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:12.963316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b539bf6ef01e43c3887a07e539e66d99)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:12.965327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:13.082542 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b539bf6ef01e43c3887a07e539e66d99', 'request_id': 'req-86c4db8f-ceff-47f0-a69b-47c3502b3400'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.150620 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b539bf6ef01e43c3887a07e539e66d99', 'request_id': 'req-86c4db8f-ceff-47f0-a69b-47c3502b3400'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.156612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b539bf6ef01e43c3887a07e539e66d99', 'request_id': 'req-86c4db8f-ceff-47f0-a69b-47c3502b3400'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.167900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] The token cache is being invalidate because project b539bf6ef01e43c3887a07e539e66d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:11:13.168194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.168820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.170346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1257/2478] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:11:12 2022] DELETE /identity/v3/projects/b539bf6ef01e43c3887a07e539e66d99 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:13.175354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:13.185062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27c435-07f4-4d07-a1d3-23e504532cc5 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:13.216090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:13.245061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:13.245709 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:13.246000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:13.246278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] PATH_INFO: `/v3/users/ba7e7d2c8e3f4addb9b069e33f5921a8` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:13.247429 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7e7d2c8e3f4addb9b069e33f5921a8)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:13.249415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:13.467288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7e7d2c8e3f4addb9b069e33f5921a8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.474984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7e7d2c8e3f4addb9b069e33f5921a8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.539625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7e7d2c8e3f4addb9b069e33f5921a8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.540455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7e7d2c8e3f4addb9b069e33f5921a8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.554262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-65a64362-5a24-4737-83c9-259843e84c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7e7d2c8e3f4addb9b069e33f5921a8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:13.560278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1222/2479] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:11:13 2022] DELETE /identity/v3/users/ba7e7d2c8e3f4addb9b069e33f5921a8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:11:13.925050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:13.927166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:13.961924 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4db8f-ceff-47f0-a69b-47c3502b3400 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:14.014811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:14.024677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:14.025325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.025612 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.026039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] PATH_INFO: `/v3/projects/79eccf08a6514edcb30424c77f0ccb38` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.027073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] RBAC: Authorizing `identity:delete_project(project_id=79eccf08a6514edcb30424c77f0ccb38)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:14.029025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:14.135335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79eccf08a6514edcb30424c77f0ccb38', 'request_id': 'req-785c62a1-a472-406b-be9d-1336f1c5fadf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:14.321652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79eccf08a6514edcb30424c77f0ccb38', 'request_id': 'req-785c62a1-a472-406b-be9d-1336f1c5fadf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:14.329295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79eccf08a6514edcb30424c77f0ccb38', 'request_id': 'req-785c62a1-a472-406b-be9d-1336f1c5fadf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:14.340229 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] The token cache is being invalidate because project 79eccf08a6514edcb30424c77f0ccb38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:11:14.340527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:14.341181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-785c62a1-a472-406b-be9d-1336f1c5fadf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:11:14.342809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1258/2480] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:11:13 2022] DELETE /identity/v3/projects/79eccf08a6514edcb30424c77f0ccb38 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:14.347991 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293bd94-c661-43b9-9046-d77364c4ac52 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.348286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293bd94-c661-43b9-9046-d77364c4ac52 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.348570 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293bd94-c661-43b9-9046-d77364c4ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.368943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e293bd94-c661-43b9-9046-d77364c4ac52 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:14.370160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e293bd94-c661-43b9-9046-d77364c4ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:14.429556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1223/2481] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:14.437367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced29d5-e51a-488a-bee5-ee89a31db20a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.440349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced29d5-e51a-488a-bee5-ee89a31db20a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.440349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced29d5-e51a-488a-bee5-ee89a31db20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.458891 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fced29d5-e51a-488a-bee5-ee89a31db20a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:14.460136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fced29d5-e51a-488a-bee5-ee89a31db20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:14.468846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1259/2482] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:14.474241 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293bd94-c661-43b9-9046-d77364c4ac52 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:14.483515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293bd94-c661-43b9-9046-d77364c4ac52 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:14.488784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:14.489405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.489844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.490147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.490846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:11:14.491159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:14.492879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:14.505673 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1224/2483] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:11:14.509949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced29d5-e51a-488a-bee5-ee89a31db20a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:14.519125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced29d5-e51a-488a-bee5-ee89a31db20a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:14.524893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:14.525857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.526147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.526457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.527127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:14.529065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:14.656559 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1260/2484] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:11:14 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:14.661234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:14.671493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffa936-f143-4f31-9be5-8c5333a99ac3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:14.705338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:14.729437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:14.730054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.730342 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.734227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.734227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:14.734227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:14.837108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1225/2485] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:11:14 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:14.841281 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:14.852990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1773812-6b0c-4c65-8d36-049414444b0a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:14.858084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:14.858710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.858998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.859280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.860025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:14.862251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:14.873457 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1261/2486] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:14.877553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:14.887944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fcfcd-559d-4392-859b-2cd98a28e759 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:14.892708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d8eb6-a4c1-4759-bca8-e726a8826b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:14.893335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d8eb6-a4c1-4759-bca8-e726a8826b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.893623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d8eb6-a4c1-4759-bca8-e726a8826b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.893905 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d8eb6-a4c1-4759-bca8-e726a8826b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.894609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d8eb6-a4c1-4759-bca8-e726a8826b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:14.896279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d8eb6-a4c1-4759-bca8-e726a8826b44 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:14.906649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1226/2487] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:14.913418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:14.922813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299dd12-584e-4957-9db3-c617a18bd70a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:14.928362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:14.929027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:14.929316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:14.929598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] PATH_INFO: `/v3/projects/f38b7436a661499197f749be047b10fa/users/180e7de3573746429afe61c1baa231e9/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:14.945513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38b7436a661499197f749be047b10fa, user_id=180e7de3573746429afe61c1baa231e9, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:14.948434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.014305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1262/2488] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:11:14 2022] PUT /identity/v3/projects/f38b7436a661499197f749be047b10fa/users/180e7de3573746429afe61c1baa231e9/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:15.025704 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe821c-666e-4130-8647-8066df519158 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.025997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe821c-666e-4130-8647-8066df519158 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.026276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe821c-666e-4130-8647-8066df519158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.060356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.062813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:15.069179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-41fe821c-666e-4130-8647-8066df519158 None None] MFA Rules not processed for user `180e7de3573746429afe61c1baa231e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:15.069583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-41fe821c-666e-4130-8647-8066df519158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:15.141806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1227/2489] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:15.142506 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8571-e0bc-4617-ba12-f854e08487c9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.150692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe821c-666e-4130-8647-8066df519158 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.157560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.157560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.157560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.157560 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.157951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:15.167409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:15.169090 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.181470 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1263/2490] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:11:15.241323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe821c-666e-4130-8647-8066df519158 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.246186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.246810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.247095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.247363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.248057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:15.249951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.318961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1228/2491] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:11:15 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:15.323120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.331354 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:15.335822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccd61e-e0a8-4636-9cd2-50d10549d555 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.393430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.394205 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.394492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.394770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.395623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:15.398345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.517439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1264/2492] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:11:15 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:11:15.521349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.531974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d32cdb-aeda-412a-b07b-f84ae99fddbb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.537442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.538052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.538339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.538613 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.539300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:15.540935 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.552392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1229/2493] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:11:15.556593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.566869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9776ac4-e3df-4922-a55e-17ca9af7ecfa admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.571848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.573918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.573918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.573918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] PATH_INFO: `/v3/projects/d991482e3d384810826220cf69bc8317/users/f150e52d625c4a79a12b9cde6e867d3c/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.588039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d991482e3d384810826220cf69bc8317, user_id=f150e52d625c4a79a12b9cde6e867d3c, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:15.590710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.666476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1265/2494] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:11:15 2022] PUT /identity/v3/projects/d991482e3d384810826220cf69bc8317/users/f150e52d625c4a79a12b9cde6e867d3c/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:15.670401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.681974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c104f2-07c1-4889-8190-0b68cc0ef34e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.699109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:15.740493 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085026a-56b6-48a3-866b-015bc89259ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.741112 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085026a-56b6-48a3-866b-015bc89259ae admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.741392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085026a-56b6-48a3-866b-015bc89259ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.741664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085026a-56b6-48a3-866b-015bc89259ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.742357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085026a-56b6-48a3-866b-015bc89259ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:15.744083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085026a-56b6-48a3-866b-015bc89259ae admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.754405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1230/2495] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:11:15.759320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.770617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52968-76fc-44d1-b802-bb6a6bfeaa88 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.775382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.776088 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.776385 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.776661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] PATH_INFO: `/v3/projects/d991482e3d384810826220cf69bc8317/users/f150e52d625c4a79a12b9cde6e867d3c/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.778630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d991482e3d384810826220cf69bc8317, user_id=f150e52d625c4a79a12b9cde6e867d3c, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:15.781341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:15.793138 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1266/2496] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:11:15 2022] PUT /identity/v3/projects/d991482e3d384810826220cf69bc8317/users/f150e52d625c4a79a12b9cde6e867d3c/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:11:15.804092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.804381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.804677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:15.832012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] MFA Rules not processed for user `f150e52d625c4a79a12b9cde6e867d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:15.833208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:15.902660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1231/2497] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:11:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:15.915065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:15.916765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:15.975447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b38bf-cbf1-449b-a501-c1ed72a78dac admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:15.982722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:15.983898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:15.986313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:15.986313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:16.066260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:16.067737 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:16.076684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1267/2498] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:16.420077 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:16.421513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:16.508006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38e035-753f-40f5-8951-b5242c083828 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:16.511840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:16.512465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:16.512752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:16.513030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:16.514825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:16.682665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:16.728124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:16.729577 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:16.741875 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1232/2499] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:16.995704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:16.997181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:17.010864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:17.054630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8895edf-8752-4130-b2d0-5333cb1896c8 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:17.058373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:17.059000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:17.059291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:17.059597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:17.061356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:17.070830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:17.072355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:17.080656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1268/2500] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:11:17.152901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:17.163071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9e314-a084-4d49-921f-c91ab6cfa42c service glance] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:17.166752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb314dbb-bf2b-4e97-af85-a936c0c8c0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:17.167392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb314dbb-bf2b-4e97-af85-a936c0c8c0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:17.167701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb314dbb-bf2b-4e97-af85-a936c0c8c0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:17.167970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb314dbb-bf2b-4e97-af85-a936c0c8c0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:17.169710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fb314dbb-bf2b-4e97-af85-a936c0c8c0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:17.183786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb314dbb-bf2b-4e97-af85-a936c0c8c0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:17.185177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb314dbb-bf2b-4e97-af85-a936c0c8c0a0 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:17.196037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1233/2501] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:17.259519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:17.260946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:17.329857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8282aea-51b6-4574-9ea8-c435fb40c3e2 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:17.335476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:17.336125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:17.336416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:17.336696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:17.338464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:17.350569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:17.352213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:17.361911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1269/2502] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:19.401690 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a14d6-8fff-491f-b749-a121ea4aac5e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:19.406024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a14d6-8fff-491f-b749-a121ea4aac5e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:19.406024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a14d6-8fff-491f-b749-a121ea4aac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:19.422554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0e9a14d6-8fff-491f-b749-a121ea4aac5e None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:19.423795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0e9a14d6-8fff-491f-b749-a121ea4aac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:19.483342 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1234/2503] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:11:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:19.496882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:19.498437 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:19.569833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d922-0f4e-4f56-a852-05f5ab519ddd service placement] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:19.574971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:19.575629 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:19.575935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:19.576204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:19.585191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:19.587008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:19.595865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1270/2504] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:11:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:11:26.552496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a14d6-8fff-491f-b749-a121ea4aac5e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:26.561682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a14d6-8fff-491f-b749-a121ea4aac5e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:26.565184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a50ec9-fd85-4c9c-b146-41d9504d1f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:26.566072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a50ec9-fd85-4c9c-b146-41d9504d1f0b service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:26.566357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a50ec9-fd85-4c9c-b146-41d9504d1f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:26.566629 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a50ec9-fd85-4c9c-b146-41d9504d1f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:26.575702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a50ec9-fd85-4c9c-b146-41d9504d1f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:26.577105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a50ec9-fd85-4c9c-b146-41d9504d1f0b service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:26.586906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1235/2505] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:11:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:29.592770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=85 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:36.582274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=54 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:44.897441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:44.899463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:44.913246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d0d39-71d7-46f7-a100-b46d0fa17daa service swift] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:44.917355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:44.918102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:44.918520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:44.918874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:44.920845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:45.138443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:45.139807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:45.159067 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1271/2506] 10.222.0.48 () {60 vars in 1396 bytes} [Mon Aug 1 08:11:44 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:11:55.156419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=41 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:11:58.054740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e460178-13d9-467f-8d65-072e1c352e80 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:58.055258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e460178-13d9-467f-8d65-072e1c352e80 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:58.055361 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e460178-13d9-467f-8d65-072e1c352e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:58.074457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5e460178-13d9-467f-8d65-072e1c352e80 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:11:58.075680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5e460178-13d9-467f-8d65-072e1c352e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:11:58.082910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1236/2507] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 08:11:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:11:58.105203 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:11:58.116654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08575046-0550-4b78-a8c2-0015ddc34bdc service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:11:58.121825 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462ff43-f00d-45b4-8f21-a79949b28eb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:11:58.122434 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462ff43-f00d-45b4-8f21-a79949b28eb2 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:11:58.122719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462ff43-f00d-45b4-8f21-a79949b28eb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:11:58.122995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462ff43-f00d-45b4-8f21-a79949b28eb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:11:58.131212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462ff43-f00d-45b4-8f21-a79949b28eb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:11:58.133399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462ff43-f00d-45b4-8f21-a79949b28eb2 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:11:58.141258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1272/2508] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:11:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:12:03.786026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e460178-13d9-467f-8d65-072e1c352e80 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:03.847968 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e460178-13d9-467f-8d65-072e1c352e80 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:03.852506 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:03.853148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:03.853430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:03.853705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] PATH_INFO: `/v3/users/180e7de3573746429afe61c1baa231e9` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:03.854871 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=180e7de3573746429afe61c1baa231e9)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:03.856873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:04.043967 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '180e7de3573746429afe61c1baa231e9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.050970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '180e7de3573746429afe61c1baa231e9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.115802 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '180e7de3573746429afe61c1baa231e9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.116745 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '180e7de3573746429afe61c1baa231e9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.130741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '180e7de3573746429afe61c1baa231e9'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.137011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1237/2509] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:03 2022] DELETE /identity/v3/users/180e7de3573746429afe61c1baa231e9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:04.141544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:04.142065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:04.142187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:04.160865 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:04.162105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:04.219480 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1273/2510] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:04.229028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:04.230927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:04.288173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f11497-1b6e-4837-9c8f-b82b9b878e10 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:04.291846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:04.292465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:04.293303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:04.293580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:04.302880 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:04.304281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:04.313688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1238/2511] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:04.757421 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:04.759361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:04.772879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230dbc8-6db6-43e2-8065-2456d3217a88 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:04.778333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:04.778994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:04.779455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:04.779769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] PATH_INFO: `/v3/projects/f38b7436a661499197f749be047b10fa` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:04.780974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f38b7436a661499197f749be047b10fa)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:04.782896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:04.881375 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f38b7436a661499197f749be047b10fa', 'request_id': 'req-14b11935-23f0-4def-983e-62291eea9a37'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.950684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f38b7436a661499197f749be047b10fa', 'request_id': 'req-14b11935-23f0-4def-983e-62291eea9a37'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.956832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f38b7436a661499197f749be047b10fa', 'request_id': 'req-14b11935-23f0-4def-983e-62291eea9a37'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.968860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] The token cache is being invalidate because project f38b7436a661499197f749be047b10fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:04.969151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.969781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:04.971311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1274/2512] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:04 2022] DELETE /identity/v3/projects/f38b7436a661499197f749be047b10fa => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:04.975392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:04.985164 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccc551-f1b7-4529-b3b0-27821bbed654 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:04.996573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:05.042022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:05.042706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:05.043008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:05.043292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] PATH_INFO: `/v3/users/f150e52d625c4a79a12b9cde6e867d3c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:05.044521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f150e52d625c4a79a12b9cde6e867d3c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:05.046729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:05.256035 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f150e52d625c4a79a12b9cde6e867d3c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.264041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f150e52d625c4a79a12b9cde6e867d3c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.326892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f150e52d625c4a79a12b9cde6e867d3c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.327556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f150e52d625c4a79a12b9cde6e867d3c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.340341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d647eae3-41f1-4062-9a87-dab4fe188e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f150e52d625c4a79a12b9cde6e867d3c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.346265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1239/2513] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:04 2022] DELETE /identity/v3/users/f150e52d625c4a79a12b9cde6e867d3c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:05.728666 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:05.730222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:05.748010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b11935-23f0-4def-983e-62291eea9a37 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:05.814461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:05.815124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:05.815406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:05.815716 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] PATH_INFO: `/v3/projects/d991482e3d384810826220cf69bc8317` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:05.816895 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d991482e3d384810826220cf69bc8317)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:05.818836 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:05.831515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:05.912112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd991482e3d384810826220cf69bc8317', 'request_id': 'req-e4f992c6-6e22-43a1-9848-67e4684c46b0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.955263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd991482e3d384810826220cf69bc8317', 'request_id': 'req-e4f992c6-6e22-43a1-9848-67e4684c46b0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.961456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd991482e3d384810826220cf69bc8317', 'request_id': 'req-e4f992c6-6e22-43a1-9848-67e4684c46b0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.973030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] The token cache is being invalidate because project d991482e3d384810826220cf69bc8317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:05.973327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.973956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e4f992c6-6e22-43a1-9848-67e4684c46b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:05.975442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1275/2514] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:05 2022] DELETE /identity/v3/projects/d991482e3d384810826220cf69bc8317 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:05.980417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1223-48d5-48e2-94c6-0ce59b15eac5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:05.985417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1223-48d5-48e2-94c6-0ce59b15eac5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:05.985417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1223-48d5-48e2-94c6-0ce59b15eac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.001061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c48d1223-48d5-48e2-94c6-0ce59b15eac5 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:06.002329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c48d1223-48d5-48e2-94c6-0ce59b15eac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:06.065851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1240/2515] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:06.073627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a906059-f1a1-4936-a2ef-c21645926881 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.074664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a906059-f1a1-4936-a2ef-c21645926881 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.074964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a906059-f1a1-4936-a2ef-c21645926881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.095450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1a906059-f1a1-4936-a2ef-c21645926881 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:06.096739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1a906059-f1a1-4936-a2ef-c21645926881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:06.105211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1276/2516] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:06.109529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1223-48d5-48e2-94c6-0ce59b15eac5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.120310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1223-48d5-48e2-94c6-0ce59b15eac5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:06.125877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:06.131457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.131457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.131457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.131457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:06.131457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:06.131457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:06.143008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1241/2517] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:12:06.147218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a906059-f1a1-4936-a2ef-c21645926881 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.156813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a906059-f1a1-4936-a2ef-c21645926881 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:06.161679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:06.162294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.162583 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.162861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.163543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:06.165540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:06.237977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1277/2518] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:06 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:06.242242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.253555 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec05716-dddc-4a75-8f17-2c0ba09c47d8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:06.273044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:06.312903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:06.313445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.313731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.314008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.314682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:06.316644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:06.430074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1242/2519] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:06 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:06.434950 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.445565 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d097a-6491-4d47-a7f1-0607a4608eb7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:06.450445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:06.451064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.451346 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.451653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.457192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:06.457192 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:06.464850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1278/2520] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:06.468889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.479255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bd6af-95ed-46e6-8d66-161efb394741 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:06.485191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cd176-e82f-4689-a383-50ef4c0abb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:06.488393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd176-e82f-4689-a383-50ef4c0abb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.488393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd176-e82f-4689-a383-50ef4c0abb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.488393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd176-e82f-4689-a383-50ef4c0abb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.488393 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cd176-e82f-4689-a383-50ef4c0abb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:06.489382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cd176-e82f-4689-a383-50ef4c0abb05 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:06.501839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1243/2521] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:06.506003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.516528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e8975-7a82-4f11-9225-77a78571d1ed admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:06.521231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:06.521900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.522190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.522472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] PATH_INFO: `/v3/projects/9035e963525d455788075bc54d4dd488/users/53376dfff76a4299a73609843ef0724e/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.538027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9035e963525d455788075bc54d4dd488, user_id=53376dfff76a4299a73609843ef0724e, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:06.543388 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:06.636157 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1279/2522] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:06 2022] PUT /identity/v3/projects/9035e963525d455788075bc54d4dd488/users/53376dfff76a4299a73609843ef0724e/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:06.646984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76503367-d417-428e-ba9b-a48d9ca4cac4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.647349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76503367-d417-428e-ba9b-a48d9ca4cac4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.647737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76503367-d417-428e-ba9b-a48d9ca4cac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.685173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-76503367-d417-428e-ba9b-a48d9ca4cac4 None None] MFA Rules not processed for user `53376dfff76a4299a73609843ef0724e`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:06.685173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-76503367-d417-428e-ba9b-a48d9ca4cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:06.753363 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1244/2523] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:06.758127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.821569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f92d1-2664-43d7-9398-78005c72ed5b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:06.827043 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:06.827713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:06.827999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:06.828297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:06.828937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:06.830844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:06.918730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1280/2524] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:06 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:06.923128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76503367-d417-428e-ba9b-a48d9ca4cac4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:06.935770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76503367-d417-428e-ba9b-a48d9ca4cac4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.000746 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.001431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.001712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.001983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.002683 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.004796 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.101128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1245/2525] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:06 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:07.106112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.120324 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b4675-1709-4967-a099-573117e400f1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.126610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.132540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.132540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.132540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.132540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.132540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.145899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1281/2526] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:07.149915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.160300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714018d2-ac58-47a5-a515-2146919dab11 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.167006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.168207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.168492 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.168770 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.169482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.171379 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.182503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1246/2527] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:07.187497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.199696 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffca48-8d91-4685-8988-9e53a72122a9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.209255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f99bdb-7866-4179-bd75-6c579dc88e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.210050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f99bdb-7866-4179-bd75-6c579dc88e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.210474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f99bdb-7866-4179-bd75-6c579dc88e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.210867 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f99bdb-7866-4179-bd75-6c579dc88e91 admin admin] PATH_INFO: `/v3/projects/bafebaf475db49a3ac1fa9638503f6bc/users/07c18fc04e9b41669ed3ebcbfb02967e/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.220889 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:07.228236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f99bdb-7866-4179-bd75-6c579dc88e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafebaf475db49a3ac1fa9638503f6bc, user_id=07c18fc04e9b41669ed3ebcbfb02967e, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.236080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f99bdb-7866-4179-bd75-6c579dc88e91 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.263050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.265172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:07.278448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:07.294414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1282/2528] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:07 2022] PUT /identity/v3/projects/bafebaf475db49a3ac1fa9638503f6bc/users/07c18fc04e9b41669ed3ebcbfb02967e/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:07.306960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f273df6-99e4-4096-a659-81051981d292 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.307249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f273df6-99e4-4096-a659-81051981d292 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.307523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f273df6-99e4-4096-a659-81051981d292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.338327 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6f273df6-99e4-4096-a659-81051981d292 None None] MFA Rules not processed for user `07c18fc04e9b41669ed3ebcbfb02967e`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:07.340988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6f273df6-99e4-4096-a659-81051981d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:07.357520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbc871-75a4-4ed3-ab49-806865164de2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.415362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1283/2529] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:07.416812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f273df6-99e4-4096-a659-81051981d292 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.428108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.428741 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.429097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.432619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.432619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:07.444913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.448053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.471753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1247/2530] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:07.497337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f273df6-99e4-4096-a659-81051981d292 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.503270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.503962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.504249 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.504525 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.505195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.507152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.627521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1284/2531] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:07 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:07.631840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.643763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc136d9-65af-4865-8845-d5db64c34329 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.704749 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.705380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.705665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.705939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.706611 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.708656 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.830434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1248/2532] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:07 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:07.834461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.845812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b653fa-df50-4037-9e30-bad3ae2cc2c5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.851376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.852134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.852423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.852697 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.853453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.856045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:07.870219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1285/2533] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:07.875965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:07.893110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929d48-66a6-4f20-b596-0471967cea27 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:07.898469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:07.899353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:07.899874 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:07.900277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] PATH_INFO: `/v3/projects/289f741b53534357aa3e3d1ea3aab123/users/69b7a0da0ecd4c4fb1bdcd8c3ca5d32e/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:07.918563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=289f741b53534357aa3e3d1ea3aab123, user_id=69b7a0da0ecd4c4fb1bdcd8c3ca5d32e, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:07.921530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:08.002222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:08.010021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1249/2534] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:07 2022] PUT /identity/v3/projects/289f741b53534357aa3e3d1ea3aab123/users/69b7a0da0ecd4c4fb1bdcd8c3ca5d32e/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:08.012510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:08.022449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370c60b-efe3-434e-b7f9-8fc584cc98e8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:08.069168 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:08.079430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786ad0a-e493-4f2e-9562-82271d71b39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:08.080166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ad0a-e493-4f2e-9562-82271d71b39e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:08.080358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ad0a-e493-4f2e-9562-82271d71b39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:08.080636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ad0a-e493-4f2e-9562-82271d71b39e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:08.081333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786ad0a-e493-4f2e-9562-82271d71b39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:08.082962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786ad0a-e493-4f2e-9562-82271d71b39e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:08.093601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1286/2535] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:08.097782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:08.111451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30648d58-bb23-459e-b179-0a45fae01ccf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:08.116564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:08.117239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:08.117544 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:08.117817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] PATH_INFO: `/v3/projects/289f741b53534357aa3e3d1ea3aab123/users/69b7a0da0ecd4c4fb1bdcd8c3ca5d32e/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:08.119823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289f741b53534357aa3e3d1ea3aab123, user_id=69b7a0da0ecd4c4fb1bdcd8c3ca5d32e, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:08.123021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:08.133669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1250/2536] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:08 2022] PUT /identity/v3/projects/289f741b53534357aa3e3d1ea3aab123/users/69b7a0da0ecd4c4fb1bdcd8c3ca5d32e/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:08.144733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:08.145029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:08.145320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:08.172999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] MFA Rules not processed for user `69b7a0da0ecd4c4fb1bdcd8c3ca5d32e`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:08.174198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:08.246451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1287/2537] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:08.257761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:08.259445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:08.320335 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa52f00-f8de-4b73-ab3b-245a761484e4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:08.324929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:08.325672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:08.326058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:08.326395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:08.336735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:08.338186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:08.349189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1251/2538] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:09.331493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:09.333023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:09.345008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa67986-25a7-423c-bbb5-e8dcda3e68e7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:09.348536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:09.349151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:09.349435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:09.349712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:09.404614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:09.406063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:09.415403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1288/2539] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:14.406868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:14.408287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:14.466756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a69a8e-8df4-43c6-a954-3c1ba7045bd3 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:14.470374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:14.471019 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:14.471304 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:14.471635 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:14.473384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:14.537431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:14.539006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:14.549742 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1252/2540] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:15.394148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:15.403114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124698-3215-476d-9ab6-8f4909898a5e service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:15.406991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:15.412081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:15.412081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:15.412081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:15.473946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:15.475755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:15.486237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1289/2541] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:18.272000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:18.274052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:18.286788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848b98c-2b76-4d2c-9748-eba562cccb92 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:18.291224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:18.291910 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:18.292191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:18.292464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] PATH_INFO: `/v3/users/0d2c7d95047445ee9d44640b5372e0e5` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:18.293627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2c7d95047445ee9d44640b5372e0e5)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:18.295523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:18.436564 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:18.448942 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2c7d95047445ee9d44640b5372e0e5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:18.456142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2c7d95047445ee9d44640b5372e0e5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:18.532263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2c7d95047445ee9d44640b5372e0e5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:18.533050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2c7d95047445ee9d44640b5372e0e5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:18.545197 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2c3ee77a-2bb2-4ee1-9f8c-ce3d05494c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2c7d95047445ee9d44640b5372e0e5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:18.552203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1253/2542] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:18 2022] DELETE /identity/v3/users/0d2c7d95047445ee9d44640b5372e0e5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:18.791921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:18.804959 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c1eb-0dc3-4953-a24e-690fc56be092 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:18.810289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:18.865286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:18.865938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:18.866228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:18.866509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] PATH_INFO: `/v3/projects/7054fc89ba16433bb0669bc6a99044a5` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:18.867732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7054fc89ba16433bb0669bc6a99044a5)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:18.869662 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:18.984803 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7054fc89ba16433bb0669bc6a99044a5', 'request_id': 'req-4445a1c4-9561-41f9-a4bf-2ffabecae9de'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.075413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7054fc89ba16433bb0669bc6a99044a5', 'request_id': 'req-4445a1c4-9561-41f9-a4bf-2ffabecae9de'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.081679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7054fc89ba16433bb0669bc6a99044a5', 'request_id': 'req-4445a1c4-9561-41f9-a4bf-2ffabecae9de'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.092899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] The token cache is being invalidate because project 7054fc89ba16433bb0669bc6a99044a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:19.093193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.093829 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4445a1c4-9561-41f9-a4bf-2ffabecae9de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.096013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1290/2543] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:18 2022] DELETE /identity/v3/projects/7054fc89ba16433bb0669bc6a99044a5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:19.101015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d675807-f015-4947-a6e6-003c94fa4dbe None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.101310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d675807-f015-4947-a6e6-003c94fa4dbe None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.101590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d675807-f015-4947-a6e6-003c94fa4dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.122217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5d675807-f015-4947-a6e6-003c94fa4dbe None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:19.123453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5d675807-f015-4947-a6e6-003c94fa4dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:19.184397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1254/2544] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:19.192102 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8955-ad30-4533-b457-78c74580682d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.192391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8955-ad30-4533-b457-78c74580682d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.192668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8955-ad30-4533-b457-78c74580682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.213321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-20ba8955-ad30-4533-b457-78c74580682d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:19.214516 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-20ba8955-ad30-4533-b457-78c74580682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:19.223014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1291/2545] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:19.227312 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d675807-f015-4947-a6e6-003c94fa4dbe None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.237619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d675807-f015-4947-a6e6-003c94fa4dbe None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.242275 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.242897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.243175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.243453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.244227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:19.244536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.246182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.261607 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1255/2546] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:12:19.262736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8955-ad30-4533-b457-78c74580682d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.273031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8955-ad30-4533-b457-78c74580682d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.279265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.279265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.279265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.279656 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.280335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.282841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.379878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1292/2547] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:19 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:19.384095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.394809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137430-f658-41d8-b84a-d4634ee17092 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.455394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.459218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.459515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.459825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.460508 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.462496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.483667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.485513 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:19.497996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67dc7-8abe-49f4-a07e-1655b1763125 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.502729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.503371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.503695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.503976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] PATH_INFO: `/v3/users/53376dfff76a4299a73609843ef0724e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.505149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=53376dfff76a4299a73609843ef0724e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.507064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.555737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1256/2548] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:19 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:19.560127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.570456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c452ec-2125-4d51-91c3-aede214cbcaf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.575006 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.575628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.575913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.576186 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.576949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.578444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.589546 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1257/2549] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:19.593306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.602582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134904a2-ee4c-4129-a5ad-ff7faf76286c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.607088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.607699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.607980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.608251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.608913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.610602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.620455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1258/2550] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:19.624482 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.631468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:19.635331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea5944-4b9f-4375-a7d0-ab7e70f2098a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.639730 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9fac0-f567-4715-9bf7-399b2efdbdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.640356 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9fac0-f567-4715-9bf7-399b2efdbdae admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.640641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9fac0-f567-4715-9bf7-399b2efdbdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.640920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9fac0-f567-4715-9bf7-399b2efdbdae admin admin] PATH_INFO: `/v3/projects/a2187f7d16c64ac294ecd6c8ef228f63/users/139a19677d184593a2bbd46cd6dc451a/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.655722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9fac0-f567-4715-9bf7-399b2efdbdae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2187f7d16c64ac294ecd6c8ef228f63, user_id=139a19677d184593a2bbd46cd6dc451a, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.658648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9fac0-f567-4715-9bf7-399b2efdbdae admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.683623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53376dfff76a4299a73609843ef0724e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.721101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1259/2551] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:19 2022] PUT /identity/v3/projects/a2187f7d16c64ac294ecd6c8ef228f63/users/139a19677d184593a2bbd46cd6dc451a/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:19.730926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.731319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.731583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.732838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53376dfff76a4299a73609843ef0724e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.740882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53376dfff76a4299a73609843ef0724e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.741568 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53376dfff76a4299a73609843ef0724e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.742411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9702b09b-ecc6-4017-8e40-7925c5d947fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53376dfff76a4299a73609843ef0724e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:19.744455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:19.761654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1293/2552] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:19 2022] DELETE /identity/v3/users/53376dfff76a4299a73609843ef0724e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:19.761878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] MFA Rules not processed for user `139a19677d184593a2bbd46cd6dc451a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:19.763661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:19.766000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.767627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.767627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.786741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:19.788000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:19.832161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1260/2553] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:19.845387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.846862 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:19.850472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1294/2554] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:19.860163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:19.861797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:19.919171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c694d43-2389-4e0f-a136-b0d813bf6dad None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.923620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2ee18-b06b-49ee-b4f0-b05b382cd546 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:19.923947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.924398 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.924707 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.924980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.929381 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103ccf9-9170-403e-b96a-579e3308ec17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:19.930003 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103ccf9-9170-403e-b96a-579e3308ec17 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:19.930290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103ccf9-9170-403e-b96a-579e3308ec17 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:19.930570 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103ccf9-9170-403e-b96a-579e3308ec17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:19.934051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.935551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.938481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103ccf9-9170-403e-b96a-579e3308ec17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:19.940623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103ccf9-9170-403e-b96a-579e3308ec17 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:19.945523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1295/2555] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:19.949941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1261/2556] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:20.171031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:20.171031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:20.191632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ee4da-8e51-4cce-998d-8c71f3f2070f service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:20.196584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:20.197234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:20.197526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:20.197807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] PATH_INFO: `/v3/users/139a19677d184593a2bbd46cd6dc451a` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:20.198978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=139a19677d184593a2bbd46cd6dc451a)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:20.200996 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:20.376441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '139a19677d184593a2bbd46cd6dc451a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.446971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '139a19677d184593a2bbd46cd6dc451a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.455864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '139a19677d184593a2bbd46cd6dc451a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.456524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '139a19677d184593a2bbd46cd6dc451a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.457130 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '139a19677d184593a2bbd46cd6dc451a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.470321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:20.475429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1296/2557] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:20 2022] DELETE /identity/v3/users/139a19677d184593a2bbd46cd6dc451a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:20.480839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:20.481522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:20.481662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:20.481915 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:20.483108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:20.520654 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:20.521979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:20.569710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04c813-e121-4f27-9307-50ee87d71b25 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:20.575724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:20.576381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:20.576664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:20.576982 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] PATH_INFO: `/v3/projects/9035e963525d455788075bc54d4dd488` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:20.578152 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9035e963525d455788075bc54d4dd488)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:20.580084 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:20.580909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:20.594403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1262/2558] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:20.605284 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:20.606940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:20.667491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeb275-7c34-4543-922a-53ddfa8baa9a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:20.671474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9035e963525d455788075bc54d4dd488', 'request_id': 'req-72b76fd8-1691-4d1a-9544-6dfc11e76201'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.720102 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:20.720729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:20.721010 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:20.721279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:20.738319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9035e963525d455788075bc54d4dd488', 'request_id': 'req-72b76fd8-1691-4d1a-9544-6dfc11e76201'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.744966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9035e963525d455788075bc54d4dd488', 'request_id': 'req-72b76fd8-1691-4d1a-9544-6dfc11e76201'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.764461 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] The token cache is being invalidate because project 9035e963525d455788075bc54d4dd488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:20.764743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.765481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:20.767337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1297/2559] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:20 2022] DELETE /identity/v3/projects/9035e963525d455788075bc54d4dd488 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:20.771342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:20.773104 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:20.799263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:20.800887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:20.811674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1263/2560] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:20.840445 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76fd8-1691-4d1a-9544-6dfc11e76201 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:20.846276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:20.847006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:20.847360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:20.847741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] PATH_INFO: `/v3/users/07c18fc04e9b41669ed3ebcbfb02967e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:20.849476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c18fc04e9b41669ed3ebcbfb02967e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:20.851190 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:21.041648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:21.056490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c18fc04e9b41669ed3ebcbfb02967e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.178145 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c18fc04e9b41669ed3ebcbfb02967e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.185964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c18fc04e9b41669ed3ebcbfb02967e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.186598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c18fc04e9b41669ed3ebcbfb02967e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.187195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8c0f127-4e22-4f8a-9158-39ffe56b94a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c18fc04e9b41669ed3ebcbfb02967e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.204869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1298/2561] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:20 2022] DELETE /identity/v3/users/07c18fc04e9b41669ed3ebcbfb02967e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:21.237439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:21.239106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:21.251759 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766616b7-ccca-4086-af7b-75ef654ba632 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:21.312686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:21.313369 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.313668 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.313947 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] PATH_INFO: `/v3/projects/a2187f7d16c64ac294ecd6c8ef228f63` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.315160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2187f7d16c64ac294ecd6c8ef228f63)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:21.317116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:21.319522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:21.452012 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2187f7d16c64ac294ecd6c8ef228f63', 'request_id': 'req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.460886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2187f7d16c64ac294ecd6c8ef228f63', 'request_id': 'req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.467140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2187f7d16c64ac294ecd6c8ef228f63', 'request_id': 'req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.543400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] The token cache is being invalidate because project a2187f7d16c64ac294ecd6c8ef228f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:21.544129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.544496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea367b02-4d92-4ae5-b7bb-6da5c9fe6c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.546376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1264/2562] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:21 2022] DELETE /identity/v3/projects/a2187f7d16c64ac294ecd6c8ef228f63 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:21.551343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.551791 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.552070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.572518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:21.573771 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:21.637957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1299/2563] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:21.646795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09330c-0c90-4544-b3cc-215eb4834811 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.647573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09330c-0c90-4544-b3cc-215eb4834811 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.647573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09330c-0c90-4544-b3cc-215eb4834811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.660818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:21.662322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:21.668675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1f09330c-0c90-4544-b3cc-215eb4834811 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:21.669897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1f09330c-0c90-4544-b3cc-215eb4834811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:21.675576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24a357-5b75-45f9-86f8-f09325491c26 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:21.679184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1265/2564] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:21.681725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:21.687724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.687724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.687724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] PATH_INFO: `/v3/projects/bafebaf475db49a3ac1fa9638503f6bc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.687724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafebaf475db49a3ac1fa9638503f6bc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:21.687724 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:21.689121 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09330c-0c90-4544-b3cc-215eb4834811 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:21.698425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09330c-0c90-4544-b3cc-215eb4834811 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:21.703625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:21.704747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.705031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.705307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.706090 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:21.706324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:21.707966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:21.720626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1266/2565] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:12:21.724925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:21.734602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5f596-2ddd-439d-92fe-6df2afc8d6ab admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:21.739620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:21.740218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.740504 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.740781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.741437 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:21.743450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:21.796481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafebaf475db49a3ac1fa9638503f6bc', 'request_id': 'req-4be40dda-ab59-4aac-b254-45ca0817df6f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.869621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafebaf475db49a3ac1fa9638503f6bc', 'request_id': 'req-4be40dda-ab59-4aac-b254-45ca0817df6f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.872053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1267/2566] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:21 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:21.878836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:21.879236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafebaf475db49a3ac1fa9638503f6bc', 'request_id': 'req-4be40dda-ab59-4aac-b254-45ca0817df6f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.898750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] The token cache is being invalidate because project bafebaf475db49a3ac1fa9638503f6bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:21.899222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.900270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:21.902723 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1300/2567] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:21 2022] DELETE /identity/v3/projects/bafebaf475db49a3ac1fa9638503f6bc => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:21.907008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:21.909134 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:21.947212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:21.960714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ae5df-ce00-46df-b9ea-88d27b23dfc1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:21.963843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:21.964469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.964755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.965032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.965738 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:21.967689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:21.978519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be40dda-ab59-4aac-b254-45ca0817df6f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:21.984404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:21.985037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:21.985430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:21.985725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] PATH_INFO: `/v3/users/69b7a0da0ecd4c4fb1bdcd8c3ca5d32e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:21.986949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b7a0da0ecd4c4fb1bdcd8c3ca5d32e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:21.988904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.064755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1268/2568] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:21 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:22.068717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.070334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:22.082762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2b105-03cf-48e0-8af7-666c6b00d09c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.087516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.088133 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.088419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.088696 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.089382 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.091062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.101351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1269/2569] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:22.105240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.114778 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dfd88-ac6c-4959-b5ed-3e6cb6e85882 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.119345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.119961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.120243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.120517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.125125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.125125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.133678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1270/2570] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:22.137465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.147700 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd221-c9eb-407e-93aa-0b39defdf67d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.153851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11496ff5-eb84-43be-91e4-c280dfd12aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.153851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11496ff5-eb84-43be-91e4-c280dfd12aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.153851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11496ff5-eb84-43be-91e4-c280dfd12aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.153851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11496ff5-eb84-43be-91e4-c280dfd12aac admin admin] PATH_INFO: `/v3/projects/6b3be8b631364f52beeacfd4d859b0f2/users/7633ce3018144c7b9eb9297a0309f047/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.170222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11496ff5-eb84-43be-91e4-c280dfd12aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3be8b631364f52beeacfd4d859b0f2, user_id=7633ce3018144c7b9eb9297a0309f047, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.173740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11496ff5-eb84-43be-91e4-c280dfd12aac admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.176027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:22.197977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b7a0da0ecd4c4fb1bdcd8c3ca5d32e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.281558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1271/2571] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:22 2022] PUT /identity/v3/projects/6b3be8b631364f52beeacfd4d859b0f2/users/7633ce3018144c7b9eb9297a0309f047/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:22.292017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16091-c016-4bad-a309-83d09928516b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.292550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16091-c016-4bad-a309-83d09928516b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.292675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b7a0da0ecd4c4fb1bdcd8c3ca5d32e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.293059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16091-c016-4bad-a309-83d09928516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.301279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b7a0da0ecd4c4fb1bdcd8c3ca5d32e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.303151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b7a0da0ecd4c4fb1bdcd8c3ca5d32e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.303151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b7a0da0ecd4c4fb1bdcd8c3ca5d32e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.321519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1301/2572] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:21 2022] DELETE /identity/v3/users/69b7a0da0ecd4c4fb1bdcd8c3ca5d32e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:22.328245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-16e16091-c016-4bad-a309-83d09928516b None None] MFA Rules not processed for user `7633ce3018144c7b9eb9297a0309f047`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:22.329526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-16e16091-c016-4bad-a309-83d09928516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:22.401569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1272/2573] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:22.406078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.407594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:22.473115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741af50-edf1-4b66-94a8-75ab5e3fe138 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.477876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.481619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.481619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.481619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.481619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.481997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.568511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1302/2574] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:22 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:22.572952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e16091-c016-4bad-a309-83d09928516b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.584081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e16091-c016-4bad-a309-83d09928516b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.641364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.642154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.646235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.646235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.646235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.646235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.728725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.730536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:22.744663 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1273/2575] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:22 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:22.745794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.748193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874023e-5048-4b8b-92df-c1adb1c008ff admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.753682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.754344 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.754820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.755173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] PATH_INFO: `/v3/projects/289f741b53534357aa3e3d1ea3aab123` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.756413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=289f741b53534357aa3e3d1ea3aab123)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.758381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.767921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60105820-6bca-461c-b9a4-95da1264a51e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.773708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.774445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.774737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.778557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.778557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.778557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.788444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1274/2576] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:22.792628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.803624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab402e4-d8ab-4df3-bd58-f172ea448746 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.808331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.813181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.813181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.813181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] PATH_INFO: `/v3/projects/bd426444499c4d5aaa0cc4fcfb8c230f/users/eb86839944e94bd4bbbbc921eb2aeaba/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.824966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd426444499c4d5aaa0cc4fcfb8c230f, user_id=eb86839944e94bd4bbbbc921eb2aeaba, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.827677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.829250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:22.842418 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289f741b53534357aa3e3d1ea3aab123', 'request_id': 'req-b41a7859-5970-427d-9057-e7db2dd494bd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.881216 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1275/2577] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:22 2022] PUT /identity/v3/projects/bd426444499c4d5aaa0cc4fcfb8c230f/users/eb86839944e94bd4bbbbc921eb2aeaba/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:22.885124 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.898193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289f741b53534357aa3e3d1ea3aab123', 'request_id': 'req-b41a7859-5970-427d-9057-e7db2dd494bd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.904404 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289f741b53534357aa3e3d1ea3aab123', 'request_id': 'req-b41a7859-5970-427d-9057-e7db2dd494bd'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.917738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] The token cache is being invalidate because project 289f741b53534357aa3e3d1ea3aab123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:22.918175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.918858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b41a7859-5970-427d-9057-e7db2dd494bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:22.920403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1303/2578] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:22 2022] DELETE /identity/v3/projects/289f741b53534357aa3e3d1ea3aab123 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:22.926856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60be387-ffea-4fd7-a18f-efd199afc41e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.927151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60be387-ffea-4fd7-a18f-efd199afc41e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.927432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60be387-ffea-4fd7-a18f-efd199afc41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.929325 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:22.946296 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b60be387-ffea-4fd7-a18f-efd199afc41e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:22.947495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b60be387-ffea-4fd7-a18f-efd199afc41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:22.949974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8870584-1d94-4689-b06a-fee1253ab2c9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.955614 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.956231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.956515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.963565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.963690 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1304/2579] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:22.963791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:22.963791 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:22.967776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7732e3-49e9-4bcf-8ff8-e26fd39ebb52 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.967776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7732e3-49e9-4bcf-8ff8-e26fd39ebb52 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.967776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7732e3-49e9-4bcf-8ff8-e26fd39ebb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:22.969524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1276/2580] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:22.974142 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:22.975813 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:22.986816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9e7732e3-49e9-4bcf-8ff8-e26fd39ebb52 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:22.988141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9e7732e3-49e9-4bcf-8ff8-e26fd39ebb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:22.988305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156d23d-5e33-4cbc-b8f2-2375ff07869d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:22.994799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe4716-f8ec-48db-b025-6bac4b66df6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:22.995680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe4716-f8ec-48db-b025-6bac4b66df6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:22.996046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1305/2581] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:22.996190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe4716-f8ec-48db-b025-6bac4b66df6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:22.996586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe4716-f8ec-48db-b025-6bac4b66df6d admin admin] PATH_INFO: `/v3/projects/bd426444499c4d5aaa0cc4fcfb8c230f/users/eb86839944e94bd4bbbbc921eb2aeaba/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.000074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe4716-f8ec-48db-b025-6bac4b66df6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd426444499c4d5aaa0cc4fcfb8c230f, user_id=eb86839944e94bd4bbbbc921eb2aeaba, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.002860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7732e3-49e9-4bcf-8ff8-e26fd39ebb52 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.005276 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe4716-f8ec-48db-b025-6bac4b66df6d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.019325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7732e3-49e9-4bcf-8ff8-e26fd39ebb52 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.022353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1277/2582] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:22 2022] PUT /identity/v3/projects/bd426444499c4d5aaa0cc4fcfb8c230f/users/eb86839944e94bd4bbbbc921eb2aeaba/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:23.034825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.035108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.035390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.063605 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] MFA Rules not processed for user `eb86839944e94bd4bbbbc921eb2aeaba`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:23.064853 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:23.078162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.078789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.079938 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.080223 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.080954 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:23.087923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.087923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.094291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1306/2583] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:12:23.099436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.109136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3152ac6-314e-49c2-a81d-abf22f5a4c16 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.114080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.114685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.114965 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.115237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.115955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.117848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.135130 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1278/2584] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:23.147280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.148747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:23.206965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdad82d-b0d9-459f-b278-71826c9f94a2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.209154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1307/2585] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:23 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:23.213345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.258713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.259346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.259661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.259948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.273141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:23.276298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11119ca3-1a10-4a48-9ff4-316eb3eb02e3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.281718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.282904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.283189 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.283463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.288474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.288474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.324194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.325674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.335194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1279/2586] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:23.375758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1308/2587] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:23 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:23.377301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.387734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89013346-5916-4f2b-bf37-fa99f6bade14 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.392565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.393173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.393459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.393761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.394452 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.396094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.406123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1280/2588] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:23.410578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.420391 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dd876-1817-4082-9832-cb08761a7b5c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.426031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.431919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.431919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.431919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.431919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.431919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.433505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:23.443073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1309/2589] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:23.447333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.457781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376457-4154-4c62-93e0-28668fd02da7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.462435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc5ddf-5f20-4f15-8927-5b88de8ecf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.463129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5ddf-5f20-4f15-8927-5b88de8ecf81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.463415 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5ddf-5f20-4f15-8927-5b88de8ecf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.463726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5ddf-5f20-4f15-8927-5b88de8ecf81 admin admin] PATH_INFO: `/v3/projects/1ec8946d25514103a5c64e909256683c/users/d857315f1a0d44f0b5f284226ff03229/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.479424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc5ddf-5f20-4f15-8927-5b88de8ecf81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec8946d25514103a5c64e909256683c, user_id=d857315f1a0d44f0b5f284226ff03229, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.482140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc5ddf-5f20-4f15-8927-5b88de8ecf81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.529455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.533661 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1281/2590] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:23 2022] PUT /identity/v3/projects/1ec8946d25514103a5c64e909256683c/users/d857315f1a0d44f0b5f284226ff03229/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:23.540603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7dbcf-bc1e-4ce3-9d18-644e301e2b31 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.597118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.597753 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.598038 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.598317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.600161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:23.604750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe39adb-52c7-4c56-ae6a-b1ace3c4188a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.605226 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe39adb-52c7-4c56-ae6a-b1ace3c4188a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.605349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe39adb-52c7-4c56-ae6a-b1ace3c4188a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.633488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-efe39adb-52c7-4c56-ae6a-b1ace3c4188a None None] MFA Rules not processed for user `d857315f1a0d44f0b5f284226ff03229`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:23.634748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-efe39adb-52c7-4c56-ae6a-b1ace3c4188a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:23.662065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.663515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.672624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1310/2591] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:23.704955 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1282/2592] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:23.709543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.777374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe80a2-d30b-45ee-af9b-43d19e258a2d service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.782832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.783699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.783987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.784259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.784933 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.786841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.848653 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1311/2593] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:23 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:23.852708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe39adb-52c7-4c56-ae6a-b1ace3c4188a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:23.864348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe39adb-52c7-4c56-ae6a-b1ace3c4188a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:23.922337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:23.922986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:23.923269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:23.923574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:23.924248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:23.926219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:23.936950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:24.001200 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1283/2594] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:23 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:24.009219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.020483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330f48-a3f9-433c-8d2b-ac44453f676e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.025479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.026075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.026361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.026638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.027444 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.029059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.040795 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1312/2595] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:24.046497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.057897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7d317-f7de-40e8-a905-6339a1475472 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.063044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31204b53-42ea-4779-b69b-bdfb6cfdd2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.066392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204b53-42ea-4779-b69b-bdfb6cfdd2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.066392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204b53-42ea-4779-b69b-bdfb6cfdd2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.066392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31204b53-42ea-4779-b69b-bdfb6cfdd2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.066392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204b53-42ea-4779-b69b-bdfb6cfdd2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.067941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31204b53-42ea-4779-b69b-bdfb6cfdd2a6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.078339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1284/2596] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:24.083486 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.093462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90082083-dd62-4dbf-9b72-2cdb3d822257 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.098390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.099045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.099334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.099646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] PATH_INFO: `/v3/projects/794738f650f043f9be9d97f210bd3e97/users/58fc4effd2024f679386e11a6e12b749/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.113969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=794738f650f043f9be9d97f210bd3e97, user_id=58fc4effd2024f679386e11a6e12b749, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.116658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.196888 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:24.201748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1313/2597] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:24 2022] PUT /identity/v3/projects/794738f650f043f9be9d97f210bd3e97/users/58fc4effd2024f679386e11a6e12b749/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:24.212898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083a5d-eb2a-4e5b-949c-cde79ff74026 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.213189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083a5d-eb2a-4e5b-949c-cde79ff74026 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.213470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083a5d-eb2a-4e5b-949c-cde79ff74026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.241593 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d8083a5d-eb2a-4e5b-949c-cde79ff74026 None None] MFA Rules not processed for user `58fc4effd2024f679386e11a6e12b749`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:24.242806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d8083a5d-eb2a-4e5b-949c-cde79ff74026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:24.307127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1285/2598] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:24.311478 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.372546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189bb20-21e0-4155-b799-159694a43c2a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.377084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.378230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.378680 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.378949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.379634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.381527 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.442167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1314/2599] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:24 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:24.446282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083a5d-eb2a-4e5b-949c-cde79ff74026 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.457595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083a5d-eb2a-4e5b-949c-cde79ff74026 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.514017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.514636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.514918 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.515190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.515893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.517831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.572358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1286/2600] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:24 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:24.576389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.586755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fedac1-0813-4eb7-a0ac-fb3ee3656fc9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.591331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.591987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.592269 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.592545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.593238 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.594842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.605230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1315/2601] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:24.609220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.619839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c958e6d-5d49-475a-a244-d70572bedaca admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.624456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.625110 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.625394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.625667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] PATH_INFO: `/v3/projects/be7a8a911f8e4c83ab0ceed4c7f1480c/users/a5c851f5eb7a46c78edf24026bf31433/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.640863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be7a8a911f8e4c83ab0ceed4c7f1480c, user_id=a5c851f5eb7a46c78edf24026bf31433, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.643586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.731852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1287/2602] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:24 2022] PUT /identity/v3/projects/be7a8a911f8e4c83ab0ceed4c7f1480c/users/a5c851f5eb7a46c78edf24026bf31433/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:24.735971 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.746634 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b314a5f-1c2c-43d8-a89a-dea5869f934d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.803347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52330e6-34c8-42b7-8966-2f7a4a8839c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.804001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52330e6-34c8-42b7-8966-2f7a4a8839c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.804313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52330e6-34c8-42b7-8966-2f7a4a8839c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.804595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52330e6-34c8-42b7-8966-2f7a4a8839c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.805284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52330e6-34c8-42b7-8966-2f7a4a8839c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.806907 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52330e6-34c8-42b7-8966-2f7a4a8839c3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.817605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1316/2603] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:24.821645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:24.831943 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77ed70-67c4-4660-8518-f5e3286227e8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:24.836434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:24.837088 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.837368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.837641 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] PATH_INFO: `/v3/projects/be7a8a911f8e4c83ab0ceed4c7f1480c/users/a5c851f5eb7a46c78edf24026bf31433/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.839667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be7a8a911f8e4c83ab0ceed4c7f1480c, user_id=a5c851f5eb7a46c78edf24026bf31433, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:24.842279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:24.852836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1288/2604] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:24 2022] PUT /identity/v3/projects/be7a8a911f8e4c83ab0ceed4c7f1480c/users/a5c851f5eb7a46c78edf24026bf31433/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:24.864553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dd2a2-f0c1-482e-9590-51565e2d24f9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:24.864957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dd2a2-f0c1-482e-9590-51565e2d24f9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:24.865247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dd2a2-f0c1-482e-9590-51565e2d24f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:24.892650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-923dd2a2-f0c1-482e-9590-51565e2d24f9 None None] MFA Rules not processed for user `a5c851f5eb7a46c78edf24026bf31433`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:24.895436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-923dd2a2-f0c1-482e-9590-51565e2d24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:24.952893 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:24.961704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1317/2605] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:24.973634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:25.004775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:25.030665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35974ab5-b6cd-42d5-8c59-acbe5072a998 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:25.035191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:25.035847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:25.036135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:25.036412 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:25.047189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:25.050314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:25.058837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1289/2606] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:25.192402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dd2a2-f0c1-482e-9590-51565e2d24f9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:25.202341 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dd2a2-f0c1-482e-9590-51565e2d24f9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:25.206695 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb2f35-193f-49bc-abf5-38ae2ca24a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:25.207305 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2f35-193f-49bc-abf5-38ae2ca24a1a service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:25.207630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2f35-193f-49bc-abf5-38ae2ca24a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:25.207913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2f35-193f-49bc-abf5-38ae2ca24a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:25.267136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb2f35-193f-49bc-abf5-38ae2ca24a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:25.269211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb2f35-193f-49bc-abf5-38ae2ca24a1a service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:25.278819 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1318/2607] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:33.496195 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:33.559224 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397fdf8-dc5e-4507-9530-d4a5eda6be40 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:33.563834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:33.564469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:33.564798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:33.565025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] PATH_INFO: `/v3/users/d857315f1a0d44f0b5f284226ff03229` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:33.566222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d857315f1a0d44f0b5f284226ff03229)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:33.568480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:33.747113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd857315f1a0d44f0b5f284226ff03229'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:33.756005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd857315f1a0d44f0b5f284226ff03229'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:33.806602 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd857315f1a0d44f0b5f284226ff03229'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:33.807249 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd857315f1a0d44f0b5f284226ff03229'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:33.823189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd857315f1a0d44f0b5f284226ff03229'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:33.830318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1290/2608] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:33 2022] DELETE /identity/v3/users/d857315f1a0d44f0b5f284226ff03229 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:33.835178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:33.835676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:33.835775 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:33.854773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:33.856021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:33.914844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1319/2609] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:33.924949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:33.926354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:33.983000 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:33.987310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a19812-4b9a-43b7-a5a8-0ec146bb3d72 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:33.991399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:33.992053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:33.992331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:33.992601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:34.002128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:34.003573 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:34.013847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1291/2610] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:34.425704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:34.428235 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:34.442493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0b851-0689-4117-be10-11fff63cc201 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:34.447093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:34.447768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:34.448052 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:34.448331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] PATH_INFO: `/v3/projects/1ec8946d25514103a5c64e909256683c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:34.449483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec8946d25514103a5c64e909256683c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:34.451413 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:34.572748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec8946d25514103a5c64e909256683c', 'request_id': 'req-63523c2c-bc4a-492d-af2a-5c039acc8e07'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.640086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec8946d25514103a5c64e909256683c', 'request_id': 'req-63523c2c-bc4a-492d-af2a-5c039acc8e07'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.645905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec8946d25514103a5c64e909256683c', 'request_id': 'req-63523c2c-bc4a-492d-af2a-5c039acc8e07'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.656758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] The token cache is being invalidate because project 1ec8946d25514103a5c64e909256683c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:34.657041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.657665 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.659158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1320/2611] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:34 2022] DELETE /identity/v3/projects/1ec8946d25514103a5c64e909256683c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:34.666027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:34.676526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dd3aa-988e-409f-820c-2cb714bc4877 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:34.733695 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:34.734341 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:34.734626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:34.734903 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] PATH_INFO: `/v3/users/58fc4effd2024f679386e11a6e12b749` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:34.736100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=58fc4effd2024f679386e11a6e12b749)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:34.738017 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:34.859520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58fc4effd2024f679386e11a6e12b749'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.866953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58fc4effd2024f679386e11a6e12b749'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.909016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58fc4effd2024f679386e11a6e12b749'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.909638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58fc4effd2024f679386e11a6e12b749'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.922890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58fc4effd2024f679386e11a6e12b749'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:34.929549 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1292/2612] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:34 2022] DELETE /identity/v3/users/58fc4effd2024f679386e11a6e12b749 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:35.334876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:35.336943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:35.351520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63523c2c-bc4a-492d-af2a-5c039acc8e07 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:35.364762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:35.410455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:35.411228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:35.411864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:35.412191 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] PATH_INFO: `/v3/projects/794738f650f043f9be9d97f210bd3e97` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:35.413747 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=794738f650f043f9be9d97f210bd3e97)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:35.415722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:35.514085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794738f650f043f9be9d97f210bd3e97', 'request_id': 'req-e364748d-0e8d-4f9d-9eb4-238435e43123'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.589308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794738f650f043f9be9d97f210bd3e97', 'request_id': 'req-e364748d-0e8d-4f9d-9eb4-238435e43123'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.595290 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794738f650f043f9be9d97f210bd3e97', 'request_id': 'req-e364748d-0e8d-4f9d-9eb4-238435e43123'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.606455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] The token cache is being invalidate because project 794738f650f043f9be9d97f210bd3e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:35.606725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.607361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.608904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1321/2613] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:35 2022] DELETE /identity/v3/projects/794738f650f043f9be9d97f210bd3e97 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:35.613021 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:35.615030 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:35.628167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623db30-83ad-404f-8e3e-c1da2ad5a77b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:35.687466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:35.688204 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:35.688451 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:35.688731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] PATH_INFO: `/v3/users/a5c851f5eb7a46c78edf24026bf31433` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:35.689901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c851f5eb7a46c78edf24026bf31433)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:35.691870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:35.705057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:35.899483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c851f5eb7a46c78edf24026bf31433'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.906956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c851f5eb7a46c78edf24026bf31433'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.989909 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c851f5eb7a46c78edf24026bf31433'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:35.990712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c851f5eb7a46c78edf24026bf31433'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:36.003397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bf2da69b-63ea-4182-81d8-91740604e42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c851f5eb7a46c78edf24026bf31433'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:36.009526 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1293/2614] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:35 2022] DELETE /identity/v3/users/a5c851f5eb7a46c78edf24026bf31433 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:36.413213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:36.415136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:36.429135 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364748d-0e8d-4f9d-9eb4-238435e43123 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:36.485896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:36.486541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:36.486828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:36.487108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] PATH_INFO: `/v3/projects/be7a8a911f8e4c83ab0ceed4c7f1480c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:36.488423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be7a8a911f8e4c83ab0ceed4c7f1480c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:36.490936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:36.601284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be7a8a911f8e4c83ab0ceed4c7f1480c', 'request_id': 'req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:36.602944 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:36.709078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be7a8a911f8e4c83ab0ceed4c7f1480c', 'request_id': 'req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:36.715128 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be7a8a911f8e4c83ab0ceed4c7f1480c', 'request_id': 'req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:36.727105 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] The token cache is being invalidate because project be7a8a911f8e4c83ab0ceed4c7f1480c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:36.727397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:36.728076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:36.729584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1322/2615] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:36 2022] DELETE /identity/v3/projects/be7a8a911f8e4c83ab0ceed4c7f1480c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:36.735463 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4615d4-dd0b-4921-91d5-c133a5f6db3f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:36.736210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4615d4-dd0b-4921-91d5-c133a5f6db3f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:36.736210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4615d4-dd0b-4921-91d5-c133a5f6db3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:36.756044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0c4615d4-dd0b-4921-91d5-c133a5f6db3f None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:36.757258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0c4615d4-dd0b-4921-91d5-c133a5f6db3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:36.781603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:36.783084 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:36.821785 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1294/2616] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:36.830187 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1c3e-44e6-465e-9efe-8b693f6772ed None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:36.830479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1c3e-44e6-465e-9efe-8b693f6772ed None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:36.830758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1c3e-44e6-465e-9efe-8b693f6772ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:36.850750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-169d1c3e-44e6-465e-9efe-8b693f6772ed None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:36.851196 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16debc-75b4-493f-9486-0b1a6cd0c1a9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:36.851885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-169d1c3e-44e6-465e-9efe-8b693f6772ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:36.855838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:36.856473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:36.856755 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:36.857027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] PATH_INFO: `/v3/users/7633ce3018144c7b9eb9297a0309f047` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:36.858187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7633ce3018144c7b9eb9297a0309f047)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:36.859647 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1295/2617] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:36.860197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:36.863965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d1c3e-44e6-465e-9efe-8b693f6772ed None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:36.874115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d1c3e-44e6-465e-9efe-8b693f6772ed None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:36.878804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:36.879404 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:36.879715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:36.879986 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:36.880923 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:36.881476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:36.883071 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:36.894586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1296/2618] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:12:36.899113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:36.908491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87606bcb-c3f8-4c12-96af-dd5e44d1d7d6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:36.913938 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:36.914512 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:36.914793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:36.915061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:36.915733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:36.917574 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.002184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1297/2619] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:36 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:37.011188 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.020734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7633ce3018144c7b9eb9297a0309f047'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:37.062468 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:37.078050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43204963-59fc-412d-a778-30b6dbe828ba admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.083296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.084470 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.084755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.085064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.085724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.087681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.091970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7633ce3018144c7b9eb9297a0309f047'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:37.100845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7633ce3018144c7b9eb9297a0309f047'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:37.101562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7633ce3018144c7b9eb9297a0309f047'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:37.102226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8404f6f7-fa37-44ae-83aa-a377e4faa867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7633ce3018144c7b9eb9297a0309f047'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:37.119857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1323/2620] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:36 2022] DELETE /identity/v3/users/7633ce3018144c7b9eb9297a0309f047 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:37.124236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.124528 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.124813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.143790 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:37.145042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:37.167211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1298/2621] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:37 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:37.171521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.173147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:37.185949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14dfc-a79a-4289-a72e-7b8397b4b409 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.199104 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:37.209538 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1324/2622] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:37.220091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.221564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:37.246592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.247210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.247495 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.247805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.248501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.250108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.260974 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1299/2623] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:37.264858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.275407 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd38e4f-b34d-4e73-b09a-fd55a66f23bf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.280199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.280569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91b32f-7205-4505-94b0-43bb3bb7da57 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.280795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.281081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.281357 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.282104 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.283795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.284840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f61f47-d0a9-4577-a908-28d12c21b0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.285264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f61f47-d0a9-4577-a908-28d12c21b0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.285693 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f61f47-d0a9-4577-a908-28d12c21b0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.286300 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f61f47-d0a9-4577-a908-28d12c21b0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.294793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1300/2624] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:37.297774 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f61f47-d0a9-4577-a908-28d12c21b0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.299051 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.299248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f61f47-d0a9-4577-a908-28d12c21b0d1 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.309098 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344f890-183b-4e6b-a2a4-a359023d6e4a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.309576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1325/2625] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:37.314447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.315054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.315340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.315653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] PATH_INFO: `/v3/projects/9b4f0a1b77f84ad1b4c7d9e22b03ec76/users/df3a8b563527426fa8c6da98781f12ae/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.332424 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4f0a1b77f84ad1b4c7d9e22b03ec76, user_id=df3a8b563527426fa8c6da98781f12ae, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.335642 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.401401 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1301/2626] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:37 2022] PUT /identity/v3/projects/9b4f0a1b77f84ad1b4c7d9e22b03ec76/users/df3a8b563527426fa8c6da98781f12ae/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:37.412715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc6de-358f-444e-94c6-2485502b1b3c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.413025 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc6de-358f-444e-94c6-2485502b1b3c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.413302 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc6de-358f-444e-94c6-2485502b1b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.441484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-eb1bc6de-358f-444e-94c6-2485502b1b3c None None] MFA Rules not processed for user `df3a8b563527426fa8c6da98781f12ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:37.442698 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-eb1bc6de-358f-444e-94c6-2485502b1b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:37.509828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1326/2627] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:37.515672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.581583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a6799-98fd-461b-b1af-2506e50bfedc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.587211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.587576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.588160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.588160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.588781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.590693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.676384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1302/2628] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:37 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:37.681338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc6de-358f-444e-94c6-2485502b1b3c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.692125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc6de-358f-444e-94c6-2485502b1b3c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.706318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.708272 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:37.743920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:37.760962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.761613 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.761905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.762759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.763450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.765512 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.784501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf7d48-969d-4ac8-8e28-e9fc849da675 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.790488 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.791135 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.793551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.793551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] PATH_INFO: `/v3/projects/6b3be8b631364f52beeacfd4d859b0f2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.793551 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3be8b631364f52beeacfd4d859b0f2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.794845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.837740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1327/2629] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:37 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:37.841474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.852481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1e58-a96b-4362-a48b-ec9fd5e5d6e1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.857165 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.857767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.858051 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.858325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.859012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.860651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.870799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1328/2630] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:37.874835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:37.884990 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3c966-ff44-4dff-bb8f-894d08071be5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:37.889673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:37.890309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:37.890735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:37.890926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] PATH_INFO: `/v3/projects/838b7bb8649543868f9b5a156e8ce111/users/aceb3afde4024a17b341fb392aa5b328/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:37.894166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:37.906505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=838b7bb8649543868f9b5a156e8ce111, user_id=aceb3afde4024a17b341fb392aa5b328, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:37.909261 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:37.911500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3be8b631364f52beeacfd4d859b0f2', 'request_id': 'req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:37.918969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3be8b631364f52beeacfd4d859b0f2', 'request_id': 'req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:37.924801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3be8b631364f52beeacfd4d859b0f2', 'request_id': 'req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.000250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] The token cache is being invalidate because project 6b3be8b631364f52beeacfd4d859b0f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:38.000545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.001050 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1329/2631] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:37 2022] PUT /identity/v3/projects/838b7bb8649543868f9b5a156e8ce111/users/aceb3afde4024a17b341fb392aa5b328/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:38.004069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.004069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1303/2632] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:37 2022] DELETE /identity/v3/projects/6b3be8b631364f52beeacfd4d859b0f2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:38.006733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:38.006958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:38.008153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:38.008792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:38.073960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf11734-ae7e-40a5-aa1b-cd60786bfc03 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:38.074908 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569cdc4-5cbf-4dcb-be59-6cd4ed392c06 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:38.078805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:38.079423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:38.079738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:38.079964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:38.080078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:38.080618 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:38.080730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:38.080906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:38.081184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] PATH_INFO: `/v3/users/eb86839944e94bd4bbbbc921eb2aeaba` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:38.082284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:38.082385 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb86839944e94bd4bbbbc921eb2aeaba)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:38.085173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:38.097624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1330/2633] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:38.097624 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:38.107955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b31863-7295-4b50-90f5-1f29d9445b18 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:38.112640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cdb89-6a8f-4cc4-8dbb-09c64297a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:38.113320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cdb89-6a8f-4cc4-8dbb-09c64297a96f admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:38.113606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cdb89-6a8f-4cc4-8dbb-09c64297a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:38.113885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cdb89-6a8f-4cc4-8dbb-09c64297a96f admin admin] PATH_INFO: `/v3/projects/838b7bb8649543868f9b5a156e8ce111/users/aceb3afde4024a17b341fb392aa5b328/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:38.118041 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cdb89-6a8f-4cc4-8dbb-09c64297a96f admin admin] RBAC: Authorizing `identity:create_grant(project_id=838b7bb8649543868f9b5a156e8ce111, user_id=aceb3afde4024a17b341fb392aa5b328, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:38.120750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cdb89-6a8f-4cc4-8dbb-09c64297a96f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:38.131758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1331/2634] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:38 2022] PUT /identity/v3/projects/838b7bb8649543868f9b5a156e8ce111/users/aceb3afde4024a17b341fb392aa5b328/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:38.142989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:38.143280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:38.143594 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:38.171544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] MFA Rules not processed for user `aceb3afde4024a17b341fb392aa5b328`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:38.172759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:38.244664 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1332/2635] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:38.256335 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:38.258024 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:38.274027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb86839944e94bd4bbbbc921eb2aeaba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.281362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb86839944e94bd4bbbbc921eb2aeaba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.283108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:38.314692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6644e1-e5d2-4194-ac7e-f3dc5d6ee453 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:38.319208 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:38.320321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:38.320635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:38.320913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:38.328069 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:38.330280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:38.331792 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:38.341974 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1333/2636] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:38.371995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb86839944e94bd4bbbbc921eb2aeaba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.372632 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb86839944e94bd4bbbbc921eb2aeaba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.386542 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-015f3599-41c3-4d06-928b-36ba1405489e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb86839944e94bd4bbbbc921eb2aeaba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:38.392777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1304/2637] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:38 2022] DELETE /identity/v3/users/eb86839944e94bd4bbbbc921eb2aeaba => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:38.939124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.012694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f00d49-e61f-4260-8f9f-957c600cdf63 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.070623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.071266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.071576 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.071860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] PATH_INFO: `/v3/projects/bd426444499c4d5aaa0cc4fcfb8c230f` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.073014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd426444499c4d5aaa0cc4fcfb8c230f)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.074976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.196246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd426444499c4d5aaa0cc4fcfb8c230f', 'request_id': 'req-2a072f88-b015-46e5-b529-b817c67d7d33'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.263230 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd426444499c4d5aaa0cc4fcfb8c230f', 'request_id': 'req-2a072f88-b015-46e5-b529-b817c67d7d33'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.269609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd426444499c4d5aaa0cc4fcfb8c230f', 'request_id': 'req-2a072f88-b015-46e5-b529-b817c67d7d33'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.282139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] The token cache is being invalidate because project bd426444499c4d5aaa0cc4fcfb8c230f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:39.282443 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.283075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a072f88-b015-46e5-b529-b817c67d7d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.284654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1334/2638] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:38 2022] DELETE /identity/v3/projects/bd426444499c4d5aaa0cc4fcfb8c230f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:39.290648 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0eab5-1cab-4631-8ee3-3156df3a4a47 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.291095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0eab5-1cab-4631-8ee3-3156df3a4a47 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.291331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0eab5-1cab-4631-8ee3-3156df3a4a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.312040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-37a0eab5-1cab-4631-8ee3-3156df3a4a47 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:39.313243 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-37a0eab5-1cab-4631-8ee3-3156df3a4a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:39.373158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1305/2639] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:39.381381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a856c8-7a8e-4da7-a637-861dea1054a0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.381679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a856c8-7a8e-4da7-a637-861dea1054a0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.381960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a856c8-7a8e-4da7-a637-861dea1054a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.406251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c0a856c8-7a8e-4da7-a637-861dea1054a0 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:39.407511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c0a856c8-7a8e-4da7-a637-861dea1054a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:39.417713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1335/2640] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:39.424922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0eab5-1cab-4631-8ee3-3156df3a4a47 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.435201 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0eab5-1cab-4631-8ee3-3156df3a4a47 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.439817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.440425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.440705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.440978 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.441698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:39.442002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.444194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.455928 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1306/2641] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:12:39.460694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a856c8-7a8e-4da7-a637-861dea1054a0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.470181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a856c8-7a8e-4da7-a637-861dea1054a0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.475551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.476180 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.476469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.476777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.477455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.479458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.580776 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1336/2642] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:39 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:39.586924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.587075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.588649 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:39.593008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:39.601997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fece3-02b4-4aa1-998d-d8206f63ebb3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.650210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d6197-0399-4379-9065-bb774859715b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.655158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.655817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.656103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.656409 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.657081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.659353 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.659622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.660029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.660316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.660595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] PATH_INFO: `/v3/users/df3a8b563527426fa8c6da98781f12ae` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.661795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df3a8b563527426fa8c6da98781f12ae)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.663788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.764055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1337/2643] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:39 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:39.765790 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.776462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81aaf-76e1-4c3f-ba6c-1446886f42ed admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.781129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.781733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.787603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.787603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.787603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.787603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.796234 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1338/2644] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:39.800220 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.809951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9808f18-bbf1-4dc2-a0fc-0faa7ff1cb63 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.816546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.816546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.816546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.816546 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.816973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.818491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.829844 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1339/2645] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:39.834430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:39.843651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419ca6-00e5-446b-beae-327fe0678378 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:39.854226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ab7aa-f897-4c62-8384-0e5669b0b2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:39.854226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab7aa-f897-4c62-8384-0e5669b0b2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.854226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab7aa-f897-4c62-8384-0e5669b0b2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.854226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab7aa-f897-4c62-8384-0e5669b0b2a2 admin admin] PATH_INFO: `/v3/projects/86632209d02e494e8e84a7bba8e803c5/users/9755a1e1560b472c9861b610be87f927/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.864132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ab7aa-f897-4c62-8384-0e5669b0b2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86632209d02e494e8e84a7bba8e803c5, user_id=9755a1e1560b472c9861b610be87f927, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:39.867077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ab7aa-f897-4c62-8384-0e5669b0b2a2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:39.877734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df3a8b563527426fa8c6da98781f12ae'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.885364 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df3a8b563527426fa8c6da98781f12ae'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.943483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1340/2646] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:39 2022] PUT /identity/v3/projects/86632209d02e494e8e84a7bba8e803c5/users/9755a1e1560b472c9861b610be87f927/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:39.953773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.954059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.954337 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:39.960927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df3a8b563527426fa8c6da98781f12ae'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.961590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df3a8b563527426fa8c6da98781f12ae'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.963569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:39.974655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15894f2a-b023-49b8-90b0-ca7d5bab8cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df3a8b563527426fa8c6da98781f12ae'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:39.981427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1307/2647] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:39 2022] DELETE /identity/v3/users/df3a8b563527426fa8c6da98781f12ae => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:39.982550 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] MFA Rules not processed for user `9755a1e1560b472c9861b610be87f927`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:39.984001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:39.986127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:39.986539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:39.986826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.007878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:40.009106 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:40.055511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1341/2648] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:40.057096 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.058967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:40.073606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1308/2649] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:40.083109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.085168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:40.105642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:40.129606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e43708-629a-42bd-9957-53a12fb8c2c3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.134782 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.135399 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.135713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.135987 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.136890 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.139369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.145109 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb835a-3e7d-412d-82b8-3182adf7985a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.149337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.149961 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.150257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.150532 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.159881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.164477 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.174640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1309/2650] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:40.237015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1342/2651] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:40 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:40.241299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.251236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf391-2bce-46a5-9e1d-20f09b9e58bf service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.310399 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.311059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.311251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.311523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.312211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.314128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.420507 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1310/2652] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:40 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:40.424297 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.435087 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd83c96-1959-4819-8807-35c6396416f8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.440284 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.444866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.444866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.444866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.444866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.444866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.455187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1343/2653] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:40.459252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.470320 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b3baa-e3c1-4ed8-b186-3fd2f79d5b9e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.475111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.475805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.476092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.476714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] PATH_INFO: `/v3/projects/84593ec6eaa84584ae2a47020a7754eb/users/9d40689ff1eb464eb5ab8224b3021a78/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.488256 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:40.494076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=84593ec6eaa84584ae2a47020a7754eb, user_id=9d40689ff1eb464eb5ab8224b3021a78, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.496855 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.629633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1311/2654] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:40 2022] PUT /identity/v3/projects/84593ec6eaa84584ae2a47020a7754eb/users/9d40689ff1eb464eb5ab8224b3021a78/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:40.631762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.633728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:40.634095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.651156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae7ccb-505b-4814-bdb5-db2b4697154d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.704317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c1eee-2af0-4a28-8fe0-9b3ecbf179eb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.710530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.711522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.711895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.712230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.713300 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.714976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.715281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.715678 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.715973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.716253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] PATH_INFO: `/v3/projects/9b4f0a1b77f84ad1b4c7d9e22b03ec76` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.717440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4f0a1b77f84ad1b4c7d9e22b03ec76)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.719397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.725674 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1312/2655] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:40.729793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.739545 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eeab-aac6-4f16-8d3b-493fc0fdf0fc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.744722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cceb26-f500-41de-8024-cc8a66484325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.745375 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cceb26-f500-41de-8024-cc8a66484325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.745664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cceb26-f500-41de-8024-cc8a66484325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.745940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cceb26-f500-41de-8024-cc8a66484325 admin admin] PATH_INFO: `/v3/projects/84593ec6eaa84584ae2a47020a7754eb/users/9d40689ff1eb464eb5ab8224b3021a78/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.747932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cceb26-f500-41de-8024-cc8a66484325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84593ec6eaa84584ae2a47020a7754eb, user_id=9d40689ff1eb464eb5ab8224b3021a78, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.756011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cceb26-f500-41de-8024-cc8a66484325 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:40.761976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1313/2656] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:40 2022] PUT /identity/v3/projects/84593ec6eaa84584ae2a47020a7754eb/users/9d40689ff1eb464eb5ab8224b3021a78/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:40.774100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.774396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.775920 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.803250 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] MFA Rules not processed for user `9d40689ff1eb464eb5ab8224b3021a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:40.804566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:40.836797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4f0a1b77f84ad1b4c7d9e22b03ec76', 'request_id': 'req-c15e16ad-5768-49f9-8958-e51a5eaed29f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:40.874675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1314/2657] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:40.886585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.887930 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:40.897923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4f0a1b77f84ad1b4c7d9e22b03ec76', 'request_id': 'req-c15e16ad-5768-49f9-8958-e51a5eaed29f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:40.904365 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4f0a1b77f84ad1b4c7d9e22b03ec76', 'request_id': 'req-c15e16ad-5768-49f9-8958-e51a5eaed29f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:40.916166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] The token cache is being invalidate because project 9b4f0a1b77f84ad1b4c7d9e22b03ec76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:40.916463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:40.917131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:40.918712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1344/2658] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:40 2022] DELETE /identity/v3/projects/9b4f0a1b77f84ad1b4c7d9e22b03ec76 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:40.922947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:40.924458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:40.947609 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2626744-77ee-4237-9e08-39c26cc93e4f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.956372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:40.970830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:40.989688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e16ad-5768-49f9-8958-e51a5eaed29f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:40.995343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.996085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.996381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.996646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] PATH_INFO: `/v3/users/aceb3afde4024a17b341fb392aa5b328` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.996753 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:40.997280 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:40.997567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:40.997895 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:40.998002 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aceb3afde4024a17b341fb392aa5b328)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:40.999747 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:41.002701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:41.063469 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:41.064999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:41.075413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1315/2659] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:41.245741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aceb3afde4024a17b341fb392aa5b328'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:41.314688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aceb3afde4024a17b341fb392aa5b328'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:41.322232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aceb3afde4024a17b341fb392aa5b328'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:41.322855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aceb3afde4024a17b341fb392aa5b328'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:41.323459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aceb3afde4024a17b341fb392aa5b328'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:41.340519 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1345/2660] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:40 2022] DELETE /identity/v3/users/aceb3afde4024a17b341fb392aa5b328 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:41.783890 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:41.796062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26f87-1f73-47b0-a69c-8ef98abdaf43 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:41.858659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:41.859314 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:41.859639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:41.860254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] PATH_INFO: `/v3/projects/838b7bb8649543868f9b5a156e8ce111` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:41.861435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=838b7bb8649543868f9b5a156e8ce111)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:41.863388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:41.995483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838b7bb8649543868f9b5a156e8ce111', 'request_id': 'req-36cc73f9-6661-40a8-96a9-d8262c9e5f48'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.001750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838b7bb8649543868f9b5a156e8ce111', 'request_id': 'req-36cc73f9-6661-40a8-96a9-d8262c9e5f48'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.007945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838b7bb8649543868f9b5a156e8ce111', 'request_id': 'req-36cc73f9-6661-40a8-96a9-d8262c9e5f48'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.015384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:42.098353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.103454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] The token cache is being invalidate because project 838b7bb8649543868f9b5a156e8ce111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:42.103454 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.103721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.103829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36cc73f9-6661-40a8-96a9-d8262c9e5f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.103829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1316/2661] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:41 2022] DELETE /identity/v3/projects/838b7bb8649543868f9b5a156e8ce111 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:42.107181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7516-499e-409a-b1ce-4397434b4428 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.107585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7516-499e-409a-b1ce-4397434b4428 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.107881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7516-499e-409a-b1ce-4397434b4428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.113161 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12840-3284-4ce2-a133-5aaf51838385 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.127733 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ba4a7516-499e-409a-b1ce-4397434b4428 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:42.128958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ba4a7516-499e-409a-b1ce-4397434b4428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.179704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.180347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.180630 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.180909 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] PATH_INFO: `/v3/users/9755a1e1560b472c9861b610be87f927` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.182377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9755a1e1560b472c9861b610be87f927)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.184343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.196750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1317/2662] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:42.204672 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f3cd4-3df2-4e0d-87f3-1e187f1bf152 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.204970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f3cd4-3df2-4e0d-87f3-1e187f1bf152 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.205251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f3cd4-3df2-4e0d-87f3-1e187f1bf152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.225901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-485f3cd4-3df2-4e0d-87f3-1e187f1bf152 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:42.227116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-485f3cd4-3df2-4e0d-87f3-1e187f1bf152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.235718 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1318/2663] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:42.239904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f3cd4-3df2-4e0d-87f3-1e187f1bf152 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.249427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f3cd4-3df2-4e0d-87f3-1e187f1bf152 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.254206 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.254841 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.255160 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.256211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.257103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:42.257416 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.259041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.274175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1319/2664] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:12:42.278362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.288623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797bd66-58c7-4e63-8ee9-324cf626483a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.293517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.294123 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.294403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.294677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.295720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.297612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.408581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9755a1e1560b472c9861b610be87f927'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.431303 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1320/2665] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:42 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:42.435414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.480129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9755a1e1560b472c9861b610be87f927'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.487677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9755a1e1560b472c9861b610be87f927'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.488367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9755a1e1560b472c9861b610be87f927'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.489008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8814a565-2176-413c-b5b9-35845db51714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9755a1e1560b472c9861b610be87f927'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:42.490773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:42.498414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8da39d-7f9e-4427-a6e2-b286c8aea8f8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.503305 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.503954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.504232 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.504502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.505220 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.507911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.508415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1346/2666] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:42 2022] DELETE /identity/v3/users/9755a1e1560b472c9861b610be87f927 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:42.512869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.513289 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.513566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.535194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:42.535194 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.589536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1321/2667] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:42 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:42.593569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1347/2668] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:42.593721 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.595227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.602946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.604460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.607896 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6c1a-3e04-4f7b-81b2-3b61bf632679 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.612650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.613269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.613557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.613833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.614536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.616713 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.627191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1322/2669] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:42.632210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.641900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37860530-16dc-4e2b-af2f-1a6cd7b4fecc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.646558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.647163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.647450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.647764 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.648450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.650044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.661417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1323/2670] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:42.664008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da2aa8-2569-4aea-a0d1-20d76a10d785 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.666064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.668915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db3014-0a10-42ae-8132-3061a18cd672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.670314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db3014-0a10-42ae-8132-3061a18cd672 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.670712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db3014-0a10-42ae-8132-3061a18cd672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.671098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db3014-0a10-42ae-8132-3061a18cd672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.676824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce452df1-c6c2-4d50-8ffa-1f988eca6572 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.681752 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db3014-0a10-42ae-8132-3061a18cd672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.682268 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.682919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.683207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.683330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db3014-0a10-42ae-8132-3061a18cd672 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.683480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] PATH_INFO: `/v3/projects/4e5eb2bd3af8417d8e7cde386fe43335/users/c956557086f544ed9df48155b5f0789b/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.693405 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1348/2671] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:12:42.698790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5eb2bd3af8417d8e7cde386fe43335, user_id=c956557086f544ed9df48155b5f0789b, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.701560 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:42.704772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:42.790057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1324/2672] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:42 2022] PUT /identity/v3/projects/4e5eb2bd3af8417d8e7cde386fe43335/users/c956557086f544ed9df48155b5f0789b/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:12:42.802033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.803006 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.803299 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.831044 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] MFA Rules not processed for user `c956557086f544ed9df48155b5f0789b`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:42.832306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.904875 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1349/2673] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:42.917095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:42.918916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:42.977780 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95506-4c09-470b-8d31-6169be2fa00c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:42.981565 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:42.982285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:42.982567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:42.982882 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:42.992044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:42.993494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:43.002624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1325/2674] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:43.127857 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:43.129691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:43.147139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:43.202960 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d77f8-1f8a-42d2-a6a8-6fc97acbe58e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:43.208132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:43.209371 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:43.209811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:43.210172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] PATH_INFO: `/v3/projects/86632209d02e494e8e84a7bba8e803c5` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:43.211600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86632209d02e494e8e84a7bba8e803c5)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:43.213561 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:43.356047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86632209d02e494e8e84a7bba8e803c5', 'request_id': 'req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.417713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86632209d02e494e8e84a7bba8e803c5', 'request_id': 'req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.423913 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86632209d02e494e8e84a7bba8e803c5', 'request_id': 'req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.435003 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] The token cache is being invalidate because project 86632209d02e494e8e84a7bba8e803c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:43.435316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.435962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.437524 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1350/2675] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:43 2022] DELETE /identity/v3/projects/86632209d02e494e8e84a7bba8e803c5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:43.441636 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:43.454041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e320e-79b0-43ef-a7ed-4959050ff046 service glance] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:43.512883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:43.513599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:43.513887 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:43.514167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] PATH_INFO: `/v3/users/9d40689ff1eb464eb5ab8224b3021a78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:43.515327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d40689ff1eb464eb5ab8224b3021a78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:43.517279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:43.677701 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d40689ff1eb464eb5ab8224b3021a78'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.685423 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d40689ff1eb464eb5ab8224b3021a78'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.754944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d40689ff1eb464eb5ab8224b3021a78'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.756267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d40689ff1eb464eb5ab8224b3021a78'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.782885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7453030-76f6-4638-a35c-d2b5c69d8540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d40689ff1eb464eb5ab8224b3021a78'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:43.789888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1326/2676] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:12:43 2022] DELETE /identity/v3/users/9d40689ff1eb464eb5ab8224b3021a78 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:44.180871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:44.182905 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:44.198879 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1b3b8-1cf3-47f5-8868-c643c0c7f56c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:44.262532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:44.263182 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.263463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.263786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] PATH_INFO: `/v3/projects/84593ec6eaa84584ae2a47020a7754eb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.264944 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84593ec6eaa84584ae2a47020a7754eb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:44.266858 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:44.360717 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:44.363600 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84593ec6eaa84584ae2a47020a7754eb', 'request_id': 'req-3f166923-523b-4717-9d7f-976f6c74c860'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:44.405998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84593ec6eaa84584ae2a47020a7754eb', 'request_id': 'req-3f166923-523b-4717-9d7f-976f6c74c860'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:44.413039 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84593ec6eaa84584ae2a47020a7754eb', 'request_id': 'req-3f166923-523b-4717-9d7f-976f6c74c860'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:44.425246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] The token cache is being invalidate because project 84593ec6eaa84584ae2a47020a7754eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:12:44.425541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:44.426195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3f166923-523b-4717-9d7f-976f6c74c860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:12:44.428394 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1351/2677] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:12:44 2022] DELETE /identity/v3/projects/84593ec6eaa84584ae2a47020a7754eb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:44.435205 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948724-5431-4e31-b6e0-75ab0b44115c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.435705 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948724-5431-4e31-b6e0-75ab0b44115c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.435827 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948724-5431-4e31-b6e0-75ab0b44115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.455984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-af948724-5431-4e31-b6e0-75ab0b44115c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:44.457194 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-af948724-5431-4e31-b6e0-75ab0b44115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:44.469901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:44.520450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1327/2678] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:44.529244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2ba9-9d7f-437f-bece-8d34ac5fe666 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.529540 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2ba9-9d7f-437f-bece-8d34ac5fe666 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.529824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2ba9-9d7f-437f-bece-8d34ac5fe666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.551013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-13ab2ba9-9d7f-437f-bece-8d34ac5fe666 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:44.552277 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-13ab2ba9-9d7f-437f-bece-8d34ac5fe666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:44.560244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1352/2679] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:44.565333 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948724-5431-4e31-b6e0-75ab0b44115c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:44.575521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948724-5431-4e31-b6e0-75ab0b44115c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:44.580225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:44.580860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.581136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.581392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.582086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:12:44.582402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:44.584699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:44.596302 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1328/2680] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:12:44.601574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2ba9-9d7f-437f-bece-8d34ac5fe666 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:44.611433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2ba9-9d7f-437f-bece-8d34ac5fe666 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:44.616359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:44.616997 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.617308 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.617585 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.618253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:44.620312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:44.699668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1353/2681] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:12:44 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:12:44.703980 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:44.715149 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d075bcf-4312-44e6-9c77-b9e8189c6472 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:44.779863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:44.780479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.780824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.781018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.781727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:44.783712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:44.858322 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1329/2682] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:12:44 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:12:44.862411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:44.873181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d810a8-e474-4aa2-a2d9-cac462d4e276 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:44.877874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:44.878476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.878758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.879031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.879751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:44.882232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:44.893027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1354/2683] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:12:44.897820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:44.908103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fdab4-00f1-45ce-b92e-54a3f0e96a81 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:44.913191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0874e-9c09-40a4-ae94-525bff24f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:44.913859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0874e-9c09-40a4-ae94-525bff24f7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.914139 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0874e-9c09-40a4-ae94-525bff24f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.914409 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0874e-9c09-40a4-ae94-525bff24f7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.915100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0874e-9c09-40a4-ae94-525bff24f7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:44.916740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0874e-9c09-40a4-ae94-525bff24f7a2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:44.928319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1330/2684] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:12:44.932593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:44.943209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699aa50-3dd2-4939-95bb-1ec4679d82ac admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:44.948248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:44.948925 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:44.949213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:44.949491 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] PATH_INFO: `/v3/projects/5e81732f21c54f07bbd838396f6cb0d6/users/54a4d08f57a24222ac7f28c57cbe3c64/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:44.966048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e81732f21c54f07bbd838396f6cb0d6, user_id=54a4d08f57a24222ac7f28c57cbe3c64, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:44.968669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:45.046864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1355/2685] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:12:44 2022] PUT /identity/v3/projects/5e81732f21c54f07bbd838396f6cb0d6/users/54a4d08f57a24222ac7f28c57cbe3c64/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:12:45.052583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb563f-64f8-4033-98eb-6cedf4cb5389 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:45.052881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb563f-64f8-4033-98eb-6cedf4cb5389 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:45.053159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb563f-64f8-4033-98eb-6cedf4cb5389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:45.073537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-47bb563f-64f8-4033-98eb-6cedf4cb5389 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:45.074767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-47bb563f-64f8-4033-98eb-6cedf4cb5389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:45.137131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1331/2686] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:12:45.147616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:45.149314 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:45.206046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567949da-b4d5-4352-8eeb-9a3f944ab467 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:45.210756 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:45.211383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:45.211705 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:45.211983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:45.221392 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:45.222804 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:45.232500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1356/2687] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:46.391814 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be7558-f850-4c29-b5e8-cd9f70bda31f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:46.392374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be7558-f850-4c29-b5e8-cd9f70bda31f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:46.392374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be7558-f850-4c29-b5e8-cd9f70bda31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:46.424517 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c3be7558-f850-4c29-b5e8-cd9f70bda31f None None] MFA Rules not processed for user `54a4d08f57a24222ac7f28c57cbe3c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:46.425735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c3be7558-f850-4c29-b5e8-cd9f70bda31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:46.438483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:46.508710 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1332/2688] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:46.520980 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:46.522547 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:46.592072 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890090a-5923-492e-9309-88b3ff8aa562 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:46.595852 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:46.596641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:46.596922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:46.597455 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:46.608957 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:46.610474 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:46.620319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1357/2689] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:48.303136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:48.304440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:48.304440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:48.324317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] MFA Rules not processed for user `54a4d08f57a24222ac7f28c57cbe3c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:12:48.325586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:48.332606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1333/2690] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:12:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:12:48.568014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:12:48.570273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:48.582295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc21d80-3d1b-470f-a8dc-307e74b3ee7b service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:12:48.585967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8996c-0c0e-4537-8f64-fbe2a869a598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:12:48.586592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8996c-0c0e-4537-8f64-fbe2a869a598 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:12:48.586876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8996c-0c0e-4537-8f64-fbe2a869a598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:12:48.587154 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8996c-0c0e-4537-8f64-fbe2a869a598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:12:48.588926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-02b8996c-0c0e-4537-8f64-fbe2a869a598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:12:48.598759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8996c-0c0e-4537-8f64-fbe2a869a598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:12:48.600734 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8996c-0c0e-4537-8f64-fbe2a869a598 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:12:48.608625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1358/2691] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:12:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:12:58.318324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=24 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:12:58.605030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:25.764516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:25.765931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:25.779576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8245d90-c83e-4567-b3ef-9b23d32ca656 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:25.784029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:25.784659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:25.784939 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:25.785212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] PATH_INFO: `/v3/users/c956557086f544ed9df48155b5f0789b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:25.786342 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c956557086f544ed9df48155b5f0789b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:25.788299 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:25.957520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c956557086f544ed9df48155b5f0789b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:25.964832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c956557086f544ed9df48155b5f0789b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.021817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c956557086f544ed9df48155b5f0789b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.022439 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c956557086f544ed9df48155b5f0789b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.035326 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c956557086f544ed9df48155b5f0789b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.042047 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1334/2692] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:13:25 2022] DELETE /identity/v3/users/c956557086f544ed9df48155b5f0789b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:13:26.046754 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:26.047266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:26.047377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:26.065962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:26.067199 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:26.125147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1359/2693] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:13:26.134826 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:26.136246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:26.195052 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8952-7d44-416d-b94a-82bfce23e5c3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:26.198997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148dc8b-fe74-4514-9c84-7d89ce9890a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:26.199722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148dc8b-fe74-4514-9c84-7d89ce9890a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:26.199948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148dc8b-fe74-4514-9c84-7d89ce9890a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:26.200225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148dc8b-fe74-4514-9c84-7d89ce9890a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:26.209349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148dc8b-fe74-4514-9c84-7d89ce9890a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:26.210800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148dc8b-fe74-4514-9c84-7d89ce9890a4 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:26.220879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1335/2694] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:26.704241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:26.705700 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:26.721682 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e8127-3880-4cee-9923-5853db352300 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:26.730010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:26.730798 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:26.731149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:26.731495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] PATH_INFO: `/v3/projects/4e5eb2bd3af8417d8e7cde386fe43335` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:26.733395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e5eb2bd3af8417d8e7cde386fe43335)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:26.735477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:26.862851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e5eb2bd3af8417d8e7cde386fe43335', 'request_id': 'req-0ddb07f7-3612-4b4b-aa12-317b0f774577'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.930588 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e5eb2bd3af8417d8e7cde386fe43335', 'request_id': 'req-0ddb07f7-3612-4b4b-aa12-317b0f774577'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.936985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e5eb2bd3af8417d8e7cde386fe43335', 'request_id': 'req-0ddb07f7-3612-4b4b-aa12-317b0f774577'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.948729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] The token cache is being invalidate because project 4e5eb2bd3af8417d8e7cde386fe43335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:13:26.949014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.949642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0ddb07f7-3612-4b4b-aa12-317b0f774577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:26.951136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1360/2695] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:13:26 2022] DELETE /identity/v3/projects/4e5eb2bd3af8417d8e7cde386fe43335 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:13:26.956089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4be9-b34c-42b0-8253-b6f74b39b0ed None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:26.956559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4be9-b34c-42b0-8253-b6f74b39b0ed None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:26.956706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4be9-b34c-42b0-8253-b6f74b39b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:26.977240 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3bac4be9-b34c-42b0-8253-b6f74b39b0ed None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:26.978547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3bac4be9-b34c-42b0-8253-b6f74b39b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:27.041804 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1336/2696] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:13:27.049899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032acbeb-5e7c-4fad-906d-5b4ef113cb48 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.050979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032acbeb-5e7c-4fad-906d-5b4ef113cb48 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.050979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032acbeb-5e7c-4fad-906d-5b4ef113cb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.070490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-032acbeb-5e7c-4fad-906d-5b4ef113cb48 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:27.071735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-032acbeb-5e7c-4fad-906d-5b4ef113cb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:27.080000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1361/2697] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:13:27.088651 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4be9-b34c-42b0-8253-b6f74b39b0ed None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.099007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4be9-b34c-42b0-8253-b6f74b39b0ed None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.104723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.105354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.105644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.105968 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.106686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:13:27.106997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.108644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.121857 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1337/2698] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:13:27.126035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032acbeb-5e7c-4fad-906d-5b4ef113cb48 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.135632 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032acbeb-5e7c-4fad-906d-5b4ef113cb48 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.140621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.141246 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.141532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.141808 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.142495 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.144573 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.223584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1362/2699] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:13:27 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:13:27.227748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.233219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:27.238594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d6607-a0a6-4a21-a5b6-6fcc10094a40 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.298005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.298563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.298854 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.299136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.299870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.301850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.406522 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1338/2700] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:13:27 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:13:27.410722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.421453 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21697ce1-7f3b-44e6-b857-c87e1f38e1ee admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.426085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.426797 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.427079 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.427355 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.428089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.429685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.439899 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1363/2701] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:13:27.443879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.454380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d665-21a9-4aff-a2ff-156d9f8b01e6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.459059 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4551ef2-6cee-4e77-9d20-681c352cf792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.459702 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4551ef2-6cee-4e77-9d20-681c352cf792 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.459992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4551ef2-6cee-4e77-9d20-681c352cf792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.460269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4551ef2-6cee-4e77-9d20-681c352cf792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.460957 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4551ef2-6cee-4e77-9d20-681c352cf792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.463228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4551ef2-6cee-4e77-9d20-681c352cf792 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.473752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1339/2702] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:13:27.477647 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.487330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e28ba8-018d-486a-a722-703fecfb8735 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.491964 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.492610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.492896 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.493176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] PATH_INFO: `/v3/projects/1c39845babd94c3898cd13c8b2f0132e/users/1a878dfa39f546708f65af0cf6abff8c/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.507331 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c39845babd94c3898cd13c8b2f0132e, user_id=1a878dfa39f546708f65af0cf6abff8c, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.510522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.588389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1364/2703] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:13:27 2022] PUT /identity/v3/projects/1c39845babd94c3898cd13c8b2f0132e/users/1a878dfa39f546708f65af0cf6abff8c/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:13:27.603953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68299455-648b-4bab-b964-822d0b661507 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.604244 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68299455-648b-4bab-b964-822d0b661507 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.604524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68299455-648b-4bab-b964-822d0b661507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.636245 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-68299455-648b-4bab-b964-822d0b661507 None None] MFA Rules not processed for user `1a878dfa39f546708f65af0cf6abff8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:27.638162 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-68299455-648b-4bab-b964-822d0b661507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:27.712750 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1340/2704] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:13:27.723619 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.725376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:27.752059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:27.806515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed3d92-bfc3-48c7-b29c-aa76675ada6e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.811078 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb72b4-0444-4495-aa57-23666dfacbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.812294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb72b4-0444-4495-aa57-23666dfacbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.812657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb72b4-0444-4495-aa57-23666dfacbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.813001 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb72b4-0444-4495-aa57-23666dfacbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.823412 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb72b4-0444-4495-aa57-23666dfacbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.825526 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb72b4-0444-4495-aa57-23666dfacbb5 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.835643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1365/2705] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:27.848086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68299455-648b-4bab-b964-822d0b661507 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:27.853218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-68299455-648b-4bab-b964-822d0b661507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:27.861866 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68299455-648b-4bab-b964-822d0b661507 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:27.865628 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:27.866263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:27.866568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:27.866831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:27.869372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:27.945181 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:27.956442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:27.958825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:27.981516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1341/2706] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:30.628951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:30.629511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:30.629511 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:30.648076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] MFA Rules not processed for user `1a878dfa39f546708f65af0cf6abff8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:30.649298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:30.657047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1366/2707] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:13:38.049014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=9 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:40.646369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=20 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:40.707637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:40.709888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:40.791966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd13629-f72e-4c9c-99f9-95042bbd5c2d service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:40.800366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:40.801011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:40.801297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:40.801995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:40.803714 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:40.822444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:40.823034 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:40.845103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1342/2708] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:41.652759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:41.654295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:41.714979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc16400-7664-4b6c-ae89-da7df17c144d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:41.718911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:41.719564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:41.719846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:41.720126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:41.721917 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:41.802726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:41.805198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:41.817523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1367/2709] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:45.978165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:45.980634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:46.090202 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8f13d-16b1-4f40-a468-73a698463bdd service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:46.095655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:46.096296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:46.096675 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:46.096844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:46.098716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:46.124444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:46.124444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:46.125197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:46.125197 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:46.136815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969a82f-8771-41df-bc17-a3fd40ed4daf service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:46.153976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:46.153976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:46.153976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:46.153976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] PATH_INFO: `/v3/users/14e5f1b4073742b4b9bd8a48cce38341` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:46.154383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e5f1b4073742b4b9bd8a48cce38341)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:46.154485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1343/2710] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:46.155311 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:46.402762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e5f1b4073742b4b9bd8a48cce38341'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:46.467390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e5f1b4073742b4b9bd8a48cce38341'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:46.474828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e5f1b4073742b4b9bd8a48cce38341'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:46.475488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e5f1b4073742b4b9bd8a48cce38341'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:46.476123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b77d3b53-977a-4bfa-b4f9-6b99783d6a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e5f1b4073742b4b9bd8a48cce38341'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:46.493628 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1368/2711] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:13:46 2022] DELETE /identity/v3/users/14e5f1b4073742b4b9bd8a48cce38341 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:13:46.734562 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:46.747158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c05a4-ac43-49c6-9d6b-1891ef6b8a40 service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:46.824349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:46.825093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:46.825881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:46.825881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] PATH_INFO: `/v3/projects/3129476636bb43e9a14ef674c4720233` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:46.826894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3129476636bb43e9a14ef674c4720233)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:46.829428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:46.854431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:46.947190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3129476636bb43e9a14ef674c4720233', 'request_id': 'req-81793519-58a2-4241-a8d3-2ec1195f9268'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:46.954087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3129476636bb43e9a14ef674c4720233', 'request_id': 'req-81793519-58a2-4241-a8d3-2ec1195f9268'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:46.960806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3129476636bb43e9a14ef674c4720233', 'request_id': 'req-81793519-58a2-4241-a8d3-2ec1195f9268'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:47.016535 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] The token cache is being invalidate because project 3129476636bb43e9a14ef674c4720233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:13:47.017137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:47.017478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-81793519-58a2-4241-a8d3-2ec1195f9268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:13:47.023633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1344/2712] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:13:46 2022] DELETE /identity/v3/projects/3129476636bb43e9a14ef674c4720233 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:13:47.024156 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e437-0e58-44e7-9cff-bda67c4d28c8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.024449 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e437-0e58-44e7-9cff-bda67c4d28c8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.024727 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075e437-0e58-44e7-9cff-bda67c4d28c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.051071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8075e437-0e58-44e7-9cff-bda67c4d28c8 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:47.052654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8075e437-0e58-44e7-9cff-bda67c4d28c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:47.087373 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:47.125508 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1369/2713] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:13:47.133715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe58f-b463-4806-a2ff-718268944c53 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.134008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe58f-b463-4806-a2ff-718268944c53 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.134286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe58f-b463-4806-a2ff-718268944c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.154970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-dfdfe58f-b463-4806-a2ff-718268944c53 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:47.156273 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dfdfe58f-b463-4806-a2ff-718268944c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:47.164503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1345/2714] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:13:47.169411 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e437-0e58-44e7-9cff-bda67c4d28c8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:47.178840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075e437-0e58-44e7-9cff-bda67c4d28c8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:47.184426 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:47.185046 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.185334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.185616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.186356 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:13:47.186669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:47.188380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:47.201097 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1370/2715] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 08:13:47.205227 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe58f-b463-4806-a2ff-718268944c53 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:47.214590 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe58f-b463-4806-a2ff-718268944c53 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:47.219259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:47.219911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.220199 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.220481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.221143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:47.223037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:47.357288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1346/2716] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:13:47 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:13:47.357429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:47.364195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4b92b-56b1-4269-9158-58da1ae0ec88 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:47.420028 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:47.420570 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.420841 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.421116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.421772 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:47.423712 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:47.502963 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1371/2717] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:13:47 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:13:47.507270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:47.517898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec5bd4-047e-4cfb-9a11-21e294d46999 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:47.522566 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:47.523179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.523467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.523776 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.524473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:47.526068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:47.536038 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1347/2718] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:13:47.540036 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:47.550807 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68809cc3-cdfd-484e-a3fd-0f3ac46b46ca admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:47.555485 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287a697-79b1-4d7e-86af-2d3f25f910c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:47.556120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a697-79b1-4d7e-86af-2d3f25f910c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.556407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a697-79b1-4d7e-86af-2d3f25f910c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.556686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a697-79b1-4d7e-86af-2d3f25f910c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.557381 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287a697-79b1-4d7e-86af-2d3f25f910c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:47.559915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287a697-79b1-4d7e-86af-2d3f25f910c3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:47.570325 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1372/2719] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:13:47.574557 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:47.584441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6fc79-9740-4e2e-87b0-ab4761370afb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:47.589025 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:47.589779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.590062 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.590337 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] PATH_INFO: `/v3/projects/0e0d366348c8470d99ee0c1807fd6c3b/users/a067a102409a425ea52cca26794f21c6/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.605178 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0d366348c8470d99ee0c1807fd6c3b, user_id=a067a102409a425ea52cca26794f21c6, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:47.607989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:47.668870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1348/2720] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:13:47 2022] PUT /identity/v3/projects/0e0d366348c8470d99ee0c1807fd6c3b/users/a067a102409a425ea52cca26794f21c6/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:13:47.680671 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0594-af9c-439c-89b1-d1d9bd546bf2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.680904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0594-af9c-439c-89b1-d1d9bd546bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.681175 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0594-af9c-439c-89b1-d1d9bd546bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.710030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-aebf0594-af9c-439c-89b1-d1d9bd546bf2 None None] MFA Rules not processed for user `a067a102409a425ea52cca26794f21c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:47.711227 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aebf0594-af9c-439c-89b1-d1d9bd546bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:47.777136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1373/2721] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:13:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:13:47.789388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:47.791259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:47.867148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d5c11-0692-4a0f-9b5e-59dfbc502540 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:47.873430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fee6f-00f0-4e92-9eca-59792c43ed2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:47.874058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fee6f-00f0-4e92-9eca-59792c43ed2a service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:47.874348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fee6f-00f0-4e92-9eca-59792c43ed2a service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:47.874631 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fee6f-00f0-4e92-9eca-59792c43ed2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:47.882679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fee6f-00f0-4e92-9eca-59792c43ed2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:47.884351 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fee6f-00f0-4e92-9eca-59792c43ed2a service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:47.894384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1349/2722] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:53.162885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84859e75-28f8-4767-9f18-e5ddb7c6fccc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:53.163463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84859e75-28f8-4767-9f18-e5ddb7c6fccc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:53.163463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84859e75-28f8-4767-9f18-e5ddb7c6fccc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:53.164866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1374/2723] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:13:53.172118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d370a2-0fa5-4ef0-9f47-b82aac44add1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:53.172576 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d370a2-0fa5-4ef0-9f47-b82aac44add1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:53.172677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d370a2-0fa5-4ef0-9f47-b82aac44add1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:53.212960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b6d370a2-0fa5-4ef0-9f47-b82aac44add1 None None] MFA Rules not processed for user `ad3cf8a3e0a346fa867445d6b6300447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:53.214904 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b6d370a2-0fa5-4ef0-9f47-b82aac44add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:53.223736 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:53.269810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1350/2724] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:13:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:13:53.282554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84859e75-28f8-4767-9f18-e5ddb7c6fccc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:53.284291 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-84859e75-28f8-4767-9f18-e5ddb7c6fccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:53.298927 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84859e75-28f8-4767-9f18-e5ddb7c6fccc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:53.302463 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962dbd0-172e-44de-b4d9-9e07f298cfc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:53.303076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962dbd0-172e-44de-b4d9-9e07f298cfc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:53.303360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962dbd0-172e-44de-b4d9-9e07f298cfc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:53.303669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962dbd0-172e-44de-b4d9-9e07f298cfc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:53.310840 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962dbd0-172e-44de-b4d9-9e07f298cfc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:53.312310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962dbd0-172e-44de-b4d9-9e07f298cfc7 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:53.320004 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1375/2725] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:13:53.404834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a22a29-3aac-4596-98fd-c5cfe1ec2041 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:53.405158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a22a29-3aac-4596-98fd-c5cfe1ec2041 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:53.405440 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a22a29-3aac-4596-98fd-c5cfe1ec2041 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:53.406944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1351/2726] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 08:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 08:13:53.412488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:53.412850 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:53.413309 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:53.422781 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:13:53.453347 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] MFA Rules not processed for user `659b367f1ce643a68d55ff1c00d8b7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:13:53.454672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:53.465048 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1376/2727] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 08:13:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:13:53.475810 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a22a29-3aac-4596-98fd-c5cfe1ec2041 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:13:53.477489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-78a22a29-3aac-4596-98fd-c5cfe1ec2041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:13:53.491192 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a22a29-3aac-4596-98fd-c5cfe1ec2041 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:13:53.497387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:13:53.497387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:13:53.497387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:13:53.497387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:13:53.506309 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:13:53.507823 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:13:53.519060 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1352/2728] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:13:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:03.512805 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:03.514414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=23 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:08.614750 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:08.616510 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:08.722473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97561d40-33fc-4661-b8b8-d6b585ec7eb9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:08.727953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:08.728688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:08.730606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:08.730606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:08.732272 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:08.748551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:08.750312 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:08.773115 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1377/2729] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:09.226022 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:09.234755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ea22-9347-4a6c-a154-e345326e8114 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:09.238487 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b242f-1840-48d1-a35e-e2fd3aa5f036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:09.239153 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b242f-1840-48d1-a35e-e2fd3aa5f036 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:09.239414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b242f-1840-48d1-a35e-e2fd3aa5f036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:09.239722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b242f-1840-48d1-a35e-e2fd3aa5f036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:09.241537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4c1b242f-1840-48d1-a35e-e2fd3aa5f036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:09.306601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b242f-1840-48d1-a35e-e2fd3aa5f036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:09.308087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b242f-1840-48d1-a35e-e2fd3aa5f036 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:09.318769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1353/2730] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:09.607352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:09.617122 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d8b7e-eb79-49c4-bc0d-5bd7a0b7dc1c service placement] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:09.621742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:09.622383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:09.625855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:09.625855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] PATH_INFO: `/v3/users/a067a102409a425ea52cca26794f21c6` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:09.625855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a067a102409a425ea52cca26794f21c6)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:09.626330 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:09.791065 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a067a102409a425ea52cca26794f21c6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:09.867209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a067a102409a425ea52cca26794f21c6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:09.875912 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a067a102409a425ea52cca26794f21c6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:09.876623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a067a102409a425ea52cca26794f21c6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:09.877259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a067a102409a425ea52cca26794f21c6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:09.912799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1378/2731] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:09 2022] DELETE /identity/v3/users/a067a102409a425ea52cca26794f21c6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:09.919345 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:09.919667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:09.919944 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:09.959180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:09.960537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:10.055331 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1354/2732] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:10.070241 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:10.072213 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:10.141648 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa5b32-ec52-498c-adba-78162f95ae45 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:10.145483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6c31f-280f-46b3-8687-6e847c51e9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:10.146188 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6c31f-280f-46b3-8687-6e847c51e9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:10.146544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6c31f-280f-46b3-8687-6e847c51e9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:10.146892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6c31f-280f-46b3-8687-6e847c51e9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:10.156621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6c31f-280f-46b3-8687-6e847c51e9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:10.158193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6c31f-280f-46b3-8687-6e847c51e9a2 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:10.169310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1379/2733] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:10.663505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:10.667263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:10.677892 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee86c24-d05a-484f-b892-5797c14f0008 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:10.691800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:10.692624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:10.693008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:10.693456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] PATH_INFO: `/v3/projects/0e0d366348c8470d99ee0c1807fd6c3b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:10.694881 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0d366348c8470d99ee0c1807fd6c3b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:10.697247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:10.802170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0d366348c8470d99ee0c1807fd6c3b', 'request_id': 'req-e9db6afa-6499-4651-b68c-546609bde59b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:10.808906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0d366348c8470d99ee0c1807fd6c3b', 'request_id': 'req-e9db6afa-6499-4651-b68c-546609bde59b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:10.814699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0d366348c8470d99ee0c1807fd6c3b', 'request_id': 'req-e9db6afa-6499-4651-b68c-546609bde59b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:10.884627 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] The token cache is being invalidate because project 0e0d366348c8470d99ee0c1807fd6c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:10.884999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:10.885611 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e9db6afa-6499-4651-b68c-546609bde59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:10.888269 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1355/2734] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:10 2022] DELETE /identity/v3/projects/0e0d366348c8470d99ee0c1807fd6c3b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:10.893012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b343c-0c24-4b58-9ca2-c5ce4e98ac43 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:10.893499 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b343c-0c24-4b58-9ca2-c5ce4e98ac43 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:10.893618 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b343c-0c24-4b58-9ca2-c5ce4e98ac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:10.904295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:10.916037 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-524b343c-0c24-4b58-9ca2-c5ce4e98ac43 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:10.917415 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-524b343c-0c24-4b58-9ca2-c5ce4e98ac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:10.988362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1380/2735] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:10.997964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95aa2b-0cb7-4eb5-b401-3bf4b3b1d429 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:10.998261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95aa2b-0cb7-4eb5-b401-3bf4b3b1d429 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:10.998801 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95aa2b-0cb7-4eb5-b401-3bf4b3b1d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.022076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-dc95aa2b-0cb7-4eb5-b401-3bf4b3b1d429 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:11.023324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dc95aa2b-0cb7-4eb5-b401-3bf4b3b1d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:11.032339 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1356/2736] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:14:11.037567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b343c-0c24-4b58-9ca2-c5ce4e98ac43 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:11.052008 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b343c-0c24-4b58-9ca2-c5ce4e98ac43 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:11.057892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:11.058547 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:11.059918 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:11.060221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.060999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:14:11.061469 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:11.063749 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:11.086086 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1381/2737] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:14:11.089558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95aa2b-0cb7-4eb5-b401-3bf4b3b1d429 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:11.112464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95aa2b-0cb7-4eb5-b401-3bf4b3b1d429 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:11.118941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:11.119803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:11.120263 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:11.120883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.122297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:11.124448 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:11.138317 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:11.252579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1357/2738] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:11 2022] POST /identity/v3/projects => generated 405 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:14:11.253929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:11.266834 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be2086-c63f-4386-bc5c-bca398153426 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:11.384872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:11.385522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:11.385816 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:11.386099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.387293 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:11.390968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:11.474951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1382/2739] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:11 2022] POST /identity/v3/users => generated 303 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:11.482655 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:11.493708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3b059-a2d3-47a5-a94c-7d314acf284b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:11.499997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:11.500784 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:11.500901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:11.501182 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.502061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:11.508092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:11.516073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1358/2740] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:11.520489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:11.532400 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238b86f-7f54-424b-871d-2280aaaa428d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:11.538849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876b8bb-0248-4337-aec9-9e719c044df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:11.539686 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876b8bb-0248-4337-aec9-9e719c044df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:11.539979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876b8bb-0248-4337-aec9-9e719c044df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:11.540263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876b8bb-0248-4337-aec9-9e719c044df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.541668 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876b8bb-0248-4337-aec9-9e719c044df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:11.544591 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876b8bb-0248-4337-aec9-9e719c044df5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:11.563904 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1383/2741] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:11.566539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:11.577795 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826f9a-4b37-433a-8c26-7d956b1775e1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:11.583165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:11.583839 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:11.584127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:11.584538 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] PATH_INFO: `/v3/projects/9ebce3def12a4bcc8b9e189b91b803c0/users/2284aad5b3fd4512a56de4c096cfeb00/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.602932 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebce3def12a4bcc8b9e189b91b803c0, user_id=2284aad5b3fd4512a56de4c096cfeb00, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:11.606812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:11.701267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1359/2742] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:11 2022] PUT /identity/v3/projects/9ebce3def12a4bcc8b9e189b91b803c0/users/2284aad5b3fd4512a56de4c096cfeb00/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:11.706679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b0f9-14d5-40b9-bed8-df63525aa60c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:11.706983 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b0f9-14d5-40b9-bed8-df63525aa60c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:11.707357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b0f9-14d5-40b9-bed8-df63525aa60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:11.743902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-08e6b0f9-14d5-40b9-bed8-df63525aa60c None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:11.745174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-08e6b0f9-14d5-40b9-bed8-df63525aa60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:11.859616 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1384/2743] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:11.874608 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:11.878064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:12.032878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f22e-ef08-4637-bafe-4bb32ad48299 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:12.037571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:12.038229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:12.038500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:12.038949 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:12.073457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:12.073457 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:12.097528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1360/2744] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:14:14.319773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d5070-7e73-4ced-8eba-58bb1b0b26da None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:14.323262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d5070-7e73-4ced-8eba-58bb1b0b26da None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:14.323262 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d5070-7e73-4ced-8eba-58bb1b0b26da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:14.344358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:14.352322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c78d5070-7e73-4ced-8eba-58bb1b0b26da None None] MFA Rules not processed for user `2284aad5b3fd4512a56de4c096cfeb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:14.353555 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c78d5070-7e73-4ced-8eba-58bb1b0b26da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:14.417515 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1385/2745] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:14 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:14.423217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:14.433239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34ad2-7ef6-4e64-80ce-b0ae3e828954 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:14.439046 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:14.439500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:14.439940 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:14.440218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:14.440927 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:14.442964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:14.596111 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1361/2746] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:14 2022] POST /identity/v3/projects => generated 403 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:14.600219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d5070-7e73-4ced-8eba-58bb1b0b26da None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:14.612013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d5070-7e73-4ced-8eba-58bb1b0b26da None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:14.677921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:14.678842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:14.679125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:14.679429 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:14.680146 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:14.682093 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:14.787567 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1386/2747] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:14 2022] POST /identity/v3/users => generated 308 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:14.791494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:14.803289 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a614c2-c30c-4f7d-837e-7a81d74d6724 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:14.807820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:14.808445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:14.809157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:14.809458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:14.810132 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:14.811818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:14.822606 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1362/2748] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:14.827071 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:14.837706 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06517cf-12e3-43cc-b6f4-b8274d8ef7c6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:14.843005 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:14.843691 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:14.844013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:14.844298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] PATH_INFO: `/v3/projects/74871c680b564df089e969bd1eda82a6/users/0d6af55af3f84630b418343ca0a2110d/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:14.860231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74871c680b564df089e969bd1eda82a6, user_id=0d6af55af3f84630b418343ca0a2110d, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:14.869708 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:14.959978 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1387/2749] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:14 2022] PUT /identity/v3/projects/74871c680b564df089e969bd1eda82a6/users/0d6af55af3f84630b418343ca0a2110d/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:14.964004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:14.974706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c9511-191d-4145-96aa-406d76dd7158 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:15.034800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638cf38-83f8-4405-8479-6e65ab6a46c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:15.035520 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638cf38-83f8-4405-8479-6e65ab6a46c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:15.035836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638cf38-83f8-4405-8479-6e65ab6a46c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:15.036141 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638cf38-83f8-4405-8479-6e65ab6a46c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:15.038362 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638cf38-83f8-4405-8479-6e65ab6a46c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:15.038650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638cf38-83f8-4405-8479-6e65ab6a46c0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:15.050252 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1363/2750] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:15.054625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:15.065441 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751ab5-3be9-4101-9453-bb22749f1d79 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:15.070465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:15.071139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:15.071423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:15.071725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] PATH_INFO: `/v3/projects/74871c680b564df089e969bd1eda82a6/users/0d6af55af3f84630b418343ca0a2110d/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:15.073926 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74871c680b564df089e969bd1eda82a6, user_id=0d6af55af3f84630b418343ca0a2110d, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:15.077976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:15.090759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1388/2751] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:15 2022] PUT /identity/v3/projects/74871c680b564df089e969bd1eda82a6/users/0d6af55af3f84630b418343ca0a2110d/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:15.436163 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a7a28-704e-441d-9529-2e5769348877 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:15.437070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a7a28-704e-441d-9529-2e5769348877 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:15.441921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a7a28-704e-441d-9529-2e5769348877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:15.462155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fc0a7a28-704e-441d-9529-2e5769348877 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:15.463387 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fc0a7a28-704e-441d-9529-2e5769348877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:15.481438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:15.547712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1364/2752] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:14:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:14:15.672760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:15.674711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:15.762592 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:15.789928 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76698b-35f8-4b59-8ba6-9418b8acc669 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:15.795364 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957973c6-ed82-4cf6-9972-67fa3c2ae5fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:15.796045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957973c6-ed82-4cf6-9972-67fa3c2ae5fe service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:15.796339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957973c6-ed82-4cf6-9972-67fa3c2ae5fe service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:15.796851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957973c6-ed82-4cf6-9972-67fa3c2ae5fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:15.807406 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957973c6-ed82-4cf6-9972-67fa3c2ae5fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:15.809372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957973c6-ed82-4cf6-9972-67fa3c2ae5fe service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:15.820881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1389/2753] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:14:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:14:16.502934 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47fed0-f586-4538-ad48-505b66954328 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:16.503844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47fed0-f586-4538-ad48-505b66954328 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:16.504285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47fed0-f586-4538-ad48-505b66954328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:16.571228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1e47fed0-f586-4538-ad48-505b66954328 None None] MFA Rules not processed for user `0d6af55af3f84630b418343ca0a2110d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:16.572790 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e47fed0-f586-4538-ad48-505b66954328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:16.722061 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1365/2754] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:16.918333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0565678-73ac-4215-b81b-b4e7687f8cf4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:16.918333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0565678-73ac-4215-b81b-b4e7687f8cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:16.918333 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0565678-73ac-4215-b81b-b4e7687f8cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:16.946222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a0565678-73ac-4215-b81b-b4e7687f8cf4 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:16.947943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a0565678-73ac-4215-b81b-b4e7687f8cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:16.959209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1390/2755] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 08:14:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:16.976029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47fed0-f586-4538-ad48-505b66954328 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:16.987889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47fed0-f586-4538-ad48-505b66954328 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:17.001981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:17.002638 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:17.002993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:17.003475 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:17.014048 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:17.016856 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:17.026876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1366/2756] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:14:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:14:17.758225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9bc1c-9188-4410-808c-1a0c0c8240ac None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:17.763922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9bc1c-9188-4410-808c-1a0c0c8240ac None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:17.764610 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9bc1c-9188-4410-808c-1a0c0c8240ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:17.785822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e7e9bc1c-9188-4410-808c-1a0c0c8240ac None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:17.787279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e7e9bc1c-9188-4410-808c-1a0c0c8240ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:17.797303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1391/2757] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:14:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:17.810213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:17.821265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d6ab1-bec0-46cb-9f56-a82f122a8c35 service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:17.827168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:17.827639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:17.827901 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:17.828172 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:17.846528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:17.849032 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:17.860167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1367/2758] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:14:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:14:18.603623 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814a23-063c-4f0d-832e-24262409c77f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:18.604107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814a23-063c-4f0d-832e-24262409c77f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:18.604231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814a23-063c-4f0d-832e-24262409c77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:18.624725 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:18.634822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4d814a23-063c-4f0d-832e-24262409c77f None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:18.637357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4d814a23-063c-4f0d-832e-24262409c77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:18.647174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1392/2759] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 08:14:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:18.677171 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814a23-063c-4f0d-832e-24262409c77f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:18.692500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814a23-063c-4f0d-832e-24262409c77f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:18.700417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:18.706991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:18.706991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:18.706991 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:18.727228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:18.729935 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:18.730058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f5b2-1a7d-4a04-8070-66293e54e2ea service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:18.736539 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:18.737554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:18.738085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:18.738528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] PATH_INFO: `/v3/users/2284aad5b3fd4512a56de4c096cfeb00` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:18.739958 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2284aad5b3fd4512a56de4c096cfeb00)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:18.742330 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:18.745207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1393/2760] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:14:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:14:18.947836 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:19.047242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2284aad5b3fd4512a56de4c096cfeb00'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.059212 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2284aad5b3fd4512a56de4c096cfeb00'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.118769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2284aad5b3fd4512a56de4c096cfeb00'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.119818 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2284aad5b3fd4512a56de4c096cfeb00'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.135971 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2284aad5b3fd4512a56de4c096cfeb00'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.148208 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1368/2761] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:18 2022] DELETE /identity/v3/users/2284aad5b3fd4512a56de4c096cfeb00 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:19.347285 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:19.348762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:19.374936 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0ba70-3310-48e1-8873-4844b78d48f2 service swift] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:19.478722 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:19.479476 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:19.479839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:19.480270 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] PATH_INFO: `/v3/users/d4d85df24c9e4312bbe2da197f55e116` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:19.514732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d85df24c9e4312bbe2da197f55e116)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:19.516785 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:19.551251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:19.552744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:19.567103 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75463fdf-ffac-4dff-8462-4eb844195f44 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:19.572465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:19.573380 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:19.573731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:19.574708 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] PATH_INFO: `/v3/projects/9ebce3def12a4bcc8b9e189b91b803c0` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:19.576174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebce3def12a4bcc8b9e189b91b803c0)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:19.579585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:19.729493 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d85df24c9e4312bbe2da197f55e116'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.745870 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebce3def12a4bcc8b9e189b91b803c0', 'request_id': 'req-05cf3695-8c46-4b0e-95da-180e1f520f54'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.763906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebce3def12a4bcc8b9e189b91b803c0', 'request_id': 'req-05cf3695-8c46-4b0e-95da-180e1f520f54'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.771671 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebce3def12a4bcc8b9e189b91b803c0', 'request_id': 'req-05cf3695-8c46-4b0e-95da-180e1f520f54'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.799496 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d85df24c9e4312bbe2da197f55e116'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.808181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d85df24c9e4312bbe2da197f55e116'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.809066 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d85df24c9e4312bbe2da197f55e116'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.809952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d85df24c9e4312bbe2da197f55e116'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.830107 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1394/2762] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:19 2022] DELETE /identity/v3/users/d4d85df24c9e4312bbe2da197f55e116 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:19.838637 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] The token cache is being invalidate because project 9ebce3def12a4bcc8b9e189b91b803c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:19.841081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.841081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:19.841989 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1369/2763] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:19 2022] DELETE /identity/v3/projects/9ebce3def12a4bcc8b9e189b91b803c0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:19.846029 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:19.847602 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:19.870802 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:19.939976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385d9f-1216-4ff9-afe2-01f95d7fec12 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:19.945477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:19.946139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:19.946466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:19.946710 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] PATH_INFO: `/v3/users/0d6af55af3f84630b418343ca0a2110d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:19.947956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6af55af3f84630b418343ca0a2110d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:19.950531 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:20.110677 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:20.116306 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:20.127153 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6af55af3f84630b418343ca0a2110d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.193031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6af55af3f84630b418343ca0a2110d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.204494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6af55af3f84630b418343ca0a2110d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.206173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6af55af3f84630b418343ca0a2110d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.206173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6af55af3f84630b418343ca0a2110d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.223474 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3695-8c46-4b0e-95da-180e1f520f54 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:20.228282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:20.229222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:20.229518 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:20.229798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] PATH_INFO: `/v3/projects/03212885780843bf830a8f030756db97` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:20.234622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1395/2764] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:19 2022] DELETE /identity/v3/users/0d6af55af3f84630b418343ca0a2110d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:20.253041 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=03212885780843bf830a8f030756db97)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:20.255087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:20.289433 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:20.364042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03212885780843bf830a8f030756db97', 'request_id': 'req-e5018409-6648-4416-ad72-4342f00091cf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.376301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03212885780843bf830a8f030756db97', 'request_id': 'req-e5018409-6648-4416-ad72-4342f00091cf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.382859 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03212885780843bf830a8f030756db97', 'request_id': 'req-e5018409-6648-4416-ad72-4342f00091cf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.432626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] The token cache is being invalidate because project 03212885780843bf830a8f030756db97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:20.432919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.434231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.436310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1370/2765] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:20 2022] DELETE /identity/v3/projects/03212885780843bf830a8f030756db97 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:20.440442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:20.441955 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:20.538440 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bfd02-ea18-499b-9d71-5cd2993b7dfd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:20.572879 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:20.573340 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:20.586661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:20.605082 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:20.606000 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:20.606298 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:20.606579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] PATH_INFO: `/v3/users/ebd64c1ff3de4d298ba87f9298ad56fa` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:20.620902 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd64c1ff3de4d298ba87f9298ad56fa)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:20.622874 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:20.655787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5018409-6648-4416-ad72-4342f00091cf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:20.660465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:20.661113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:20.661398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:20.661680 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] PATH_INFO: `/v3/projects/74871c680b564df089e969bd1eda82a6` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:20.662860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=74871c680b564df089e969bd1eda82a6)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:20.664787 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:20.814703 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd64c1ff3de4d298ba87f9298ad56fa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.815265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74871c680b564df089e969bd1eda82a6', 'request_id': 'req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.822118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74871c680b564df089e969bd1eda82a6', 'request_id': 'req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.828426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74871c680b564df089e969bd1eda82a6', 'request_id': 'req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.889751 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd64c1ff3de4d298ba87f9298ad56fa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.896943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd64c1ff3de4d298ba87f9298ad56fa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.897564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd64c1ff3de4d298ba87f9298ad56fa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.898185 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0bb35de5-c3f9-46d9-a6d2-43d01feda219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd64c1ff3de4d298ba87f9298ad56fa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.915887 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1396/2766] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:20 2022] DELETE /identity/v3/users/ebd64c1ff3de4d298ba87f9298ad56fa => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:20.941258 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] The token cache is being invalidate because project 74871c680b564df089e969bd1eda82a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:20.941554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.942189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2bfa3cde-6404-49a6-9232-8c41e6aff5ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:20.943822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1371/2767] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:20 2022] DELETE /identity/v3/projects/74871c680b564df089e969bd1eda82a6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:20.953518 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b117d-37e6-4cc0-8125-75966da436b9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:20.953811 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b117d-37e6-4cc0-8125-75966da436b9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:20.954091 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b117d-37e6-4cc0-8125-75966da436b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:20.974481 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-985b117d-37e6-4cc0-8125-75966da436b9 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:20.975731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-985b117d-37e6-4cc0-8125-75966da436b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:21.038242 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1397/2768] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:14:21.048014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0c5a-4275-4287-bf0b-8aa42bcad321 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.048311 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0c5a-4275-4287-bf0b-8aa42bcad321 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.048592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0c5a-4275-4287-bf0b-8aa42bcad321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.069444 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4e6a0c5a-4275-4287-bf0b-8aa42bcad321 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:21.070667 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e6a0c5a-4275-4287-bf0b-8aa42bcad321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:21.079147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1372/2769] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:21.090247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b117d-37e6-4cc0-8125-75966da436b9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.100243 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b117d-37e6-4cc0-8125-75966da436b9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.105010 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.106127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.106414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.106694 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.107396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:14:21.107738 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.109357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.121861 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1398/2770] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:14:21.126072 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a0c5a-4275-4287-bf0b-8aa42bcad321 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.135318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a0c5a-4275-4287-bf0b-8aa42bcad321 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.140093 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.140706 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.144376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.144376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.144376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.144376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.172339 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.174166 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:21.191040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d7386-c8ad-41e2-a425-e21ade4895c9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.197424 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.198085 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.199545 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.204796 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] PATH_INFO: `/v3/projects/5a25e70fd5074ac9981af72d76fd0662` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.215013 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a25e70fd5074ac9981af72d76fd0662)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.217119 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.225267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1373/2771] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:21 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:21.231091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.241582 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:21.295307 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7573a8-0501-4cde-874e-33b511130675 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.297098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:21.300222 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.300851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.301138 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.301413 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.302078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.304095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.314320 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a25e70fd5074ac9981af72d76fd0662', 'request_id': 'req-53cf8acb-822b-4d50-8062-44d764a07314'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:21.380976 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a25e70fd5074ac9981af72d76fd0662', 'request_id': 'req-53cf8acb-822b-4d50-8062-44d764a07314'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:21.387253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a25e70fd5074ac9981af72d76fd0662', 'request_id': 'req-53cf8acb-822b-4d50-8062-44d764a07314'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:21.387414 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1374/2772] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:21 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:14:21.391346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.400924 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b42840-0490-4f31-bcae-ce464b898351 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.401070 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] The token cache is being invalidate because project 5a25e70fd5074ac9981af72d76fd0662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:21.403818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:21.403818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:21.403818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1399/2773] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:21 2022] DELETE /identity/v3/projects/5a25e70fd5074ac9981af72d76fd0662 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:21.471117 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.471805 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.472094 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.472373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.473156 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.474817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.487207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1375/2774] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:14:21.491450 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.492946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:21.506460 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf8acb-822b-4d50-8062-44d764a07314 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.511601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.516704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.516704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.516704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.516704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.516704 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.526787 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1400/2775] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:21.532165 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.549944 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.551941 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:21.553461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59741ee-f7ad-46d7-8cea-e8f11f1463bf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.560748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779141f-f729-466c-a183-089b870d9018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.565213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779141f-f729-466c-a183-089b870d9018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.565213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779141f-f729-466c-a183-089b870d9018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.565213 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779141f-f729-466c-a183-089b870d9018 admin admin] PATH_INFO: `/v3/projects/c1de6290280a48a8a2802a1c6c8113df/users/735e4542480140a0aa6c8c281182a1d3/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.586489 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779141f-f729-466c-a183-089b870d9018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1de6290280a48a8a2802a1c6c8113df, user_id=735e4542480140a0aa6c8c281182a1d3, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.592018 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779141f-f729-466c-a183-089b870d9018 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.657147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1376/2776] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:21 2022] PUT /identity/v3/projects/c1de6290280a48a8a2802a1c6c8113df/users/735e4542480140a0aa6c8c281182a1d3/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:21.660878 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec62f7-57c9-4f05-97be-969ec22a6199 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.668863 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fa954-ed3e-4d8f-bd4e-406ababd16a1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.673567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fa954-ed3e-4d8f-bd4e-406ababd16a1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.673567 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fa954-ed3e-4d8f-bd4e-406ababd16a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.698712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c20fa954-ed3e-4d8f-bd4e-406ababd16a1 None None] MFA Rules not processed for user `735e4542480140a0aa6c8c281182a1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:21.700358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c20fa954-ed3e-4d8f-bd4e-406ababd16a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:21.733922 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.734553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.734839 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.735120 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.736958 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:21.766999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:21.770762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1377/2777] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:21.775285 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fa954-ed3e-4d8f-bd4e-406ababd16a1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.782040 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:21.806224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.806224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.813155 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1401/2778] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:21.842004 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fa954-ed3e-4d8f-bd4e-406ababd16a1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:21.846685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:21.847266 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:21.847583 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:21.847867 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:21.848537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:21.850445 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:21.923316 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1378/2779] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:21 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:14:21.924746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:21.935606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73d3aa-3c74-48b3-bd29-541b586d3e35 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:22.013351 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:22.013988 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:22.014301 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:22.014641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:22.015265 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:22.017286 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:22.114590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1402/2780] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:21 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:14:22.118586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:22.129985 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a908-5d74-4920-8381-9aa807e446a9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:22.134793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:22.135420 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:22.135739 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:22.136073 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:22.136837 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:22.138471 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:22.150136 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1379/2781] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:22.154358 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:22.164995 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b144d41-3f7d-470c-8eac-f817e437b064 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:22.170872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:22.171659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:22.171949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:22.172232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] PATH_INFO: `/v3/projects/a0d2104094114f2dabc8d39dcee5bf28/users/c7a77aa689b5424d9b4c5b8c7665b736/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:22.188446 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d2104094114f2dabc8d39dcee5bf28, user_id=c7a77aa689b5424d9b4c5b8c7665b736, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:22.191164 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:22.261640 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1403/2782] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:22 2022] PUT /identity/v3/projects/a0d2104094114f2dabc8d39dcee5bf28/users/c7a77aa689b5424d9b4c5b8c7665b736/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:22.265752 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:22.277774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915a29-2e1f-48ae-a715-dc0514c9ba41 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:22.338091 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d300a-4e0b-486c-ac2d-ba7b8eed0f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:22.338722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d300a-4e0b-486c-ac2d-ba7b8eed0f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:22.339007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d300a-4e0b-486c-ac2d-ba7b8eed0f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:22.339278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d300a-4e0b-486c-ac2d-ba7b8eed0f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:22.340559 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d300a-4e0b-486c-ac2d-ba7b8eed0f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:22.342152 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d300a-4e0b-486c-ac2d-ba7b8eed0f93 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:22.352069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1380/2783] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:22.357016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:22.367562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb87ac3-f3f2-4944-876b-a12deaef2f3f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:22.372397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:22.373109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:22.373396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:22.373677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] PATH_INFO: `/v3/projects/a0d2104094114f2dabc8d39dcee5bf28/users/c7a77aa689b5424d9b4c5b8c7665b736/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:22.375718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d2104094114f2dabc8d39dcee5bf28, user_id=c7a77aa689b5424d9b4c5b8c7665b736, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:22.378367 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:22.390252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1404/2784] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:22 2022] PUT /identity/v3/projects/a0d2104094114f2dabc8d39dcee5bf28/users/c7a77aa689b5424d9b4c5b8c7665b736/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:22.401400 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:22.401689 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:22.402494 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:22.431365 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] MFA Rules not processed for user `c7a77aa689b5424d9b4c5b8c7665b736`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:22.432625 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:22.492503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:22.501486 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1381/2785] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:22 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:22.512338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:22.514020 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:22.572952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d49b2e-be6a-4047-8209-ba9d3848b2bd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:22.577374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d858d-1237-4105-adc9-a969494611ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:22.578015 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d858d-1237-4105-adc9-a969494611ce service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:22.578303 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d858d-1237-4105-adc9-a969494611ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:22.578584 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d858d-1237-4105-adc9-a969494611ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:22.587637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d858d-1237-4105-adc9-a969494611ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:22.589081 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d858d-1237-4105-adc9-a969494611ce service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:22.599357 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1405/2786] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:22 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:24.736977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:24.738665 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:24.832373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afab29b-c96f-45c7-97cb-8c40fc82aec0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:24.839024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:24.839024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:24.839024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:24.839024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] PATH_INFO: `/v3/users/1a878dfa39f546708f65af0cf6abff8c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:24.840921 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a878dfa39f546708f65af0cf6abff8c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:24.843653 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:24.989173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a878dfa39f546708f65af0cf6abff8c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:24.996156 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a878dfa39f546708f65af0cf6abff8c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.066267 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a878dfa39f546708f65af0cf6abff8c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.066907 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a878dfa39f546708f65af0cf6abff8c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.080026 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a878dfa39f546708f65af0cf6abff8c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.086108 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1382/2787] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:24 2022] DELETE /identity/v3/users/1a878dfa39f546708f65af0cf6abff8c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:25.090868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:25.091374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:25.091492 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:25.110435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:25.111673 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:25.140442 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:25.172557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1406/2788] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:14:25.188596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:25.188596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:25.263425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b552a-91f9-4e6c-bb96-2f0b46aa9a5c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:25.267151 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:25.267800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:25.268083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:25.268354 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:25.277288 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:25.279491 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:25.288832 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1383/2789] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:25.497638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:25.499255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:25.513642 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd6f11-b855-43b0-9d2f-2466e11432ef None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:25.518244 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:25.519021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:25.519416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:25.519866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] PATH_INFO: `/v3/users/735e4542480140a0aa6c8c281182a1d3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:25.521167 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=735e4542480140a0aa6c8c281182a1d3)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:25.523448 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:25.611419 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:25.613168 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:25.632681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38c476-67b5-43fb-9b65-8982fa5ba0f1 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:25.639031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:25.639925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:25.640154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:25.640426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] PATH_INFO: `/v3/projects/1c39845babd94c3898cd13c8b2f0132e` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:25.641678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c39845babd94c3898cd13c8b2f0132e)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:25.643807 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:25.657063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:25.675920 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '735e4542480140a0aa6c8c281182a1d3'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.752643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '735e4542480140a0aa6c8c281182a1d3'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.765557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '735e4542480140a0aa6c8c281182a1d3'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.766263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '735e4542480140a0aa6c8c281182a1d3'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.766881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-99e74ec1-92c3-44f7-bb8a-848d1b488616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '735e4542480140a0aa6c8c281182a1d3'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.775242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c39845babd94c3898cd13c8b2f0132e', 'request_id': 'req-a67240bd-fd74-44ed-94e7-9b4ea4c77343'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.782692 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c39845babd94c3898cd13c8b2f0132e', 'request_id': 'req-a67240bd-fd74-44ed-94e7-9b4ea4c77343'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.786458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1407/2790] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:25 2022] DELETE /identity/v3/users/735e4542480140a0aa6c8c281182a1d3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:25.788821 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c39845babd94c3898cd13c8b2f0132e', 'request_id': 'req-a67240bd-fd74-44ed-94e7-9b4ea4c77343'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.790871 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:25.791162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:25.791743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:25.811818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:25.813056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:25.849257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] The token cache is being invalidate because project 1c39845babd94c3898cd13c8b2f0132e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:25.849553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.850229 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:25.852086 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1384/2791] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:25 2022] DELETE /identity/v3/projects/1c39845babd94c3898cd13c8b2f0132e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:25.874099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1408/2792] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:25.890926 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:25.892459 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:25.958849 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67240bd-fd74-44ed-94e7-9b4ea4c77343 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:26.013001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:26.013860 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:26.014265 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:26.014552 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:26.025390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:26.026933 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:26.039045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1385/2793] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:26.486007 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:26.487427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:26.500713 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fffc2-9d8c-4ca4-acf5-2e85fd08cb2d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:26.505390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:26.506033 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:26.506317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:26.506593 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] PATH_INFO: `/v3/projects/c1de6290280a48a8a2802a1c6c8113df` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:26.507780 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1de6290280a48a8a2802a1c6c8113df)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:26.509701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:26.596477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:26.603855 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1de6290280a48a8a2802a1c6c8113df', 'request_id': 'req-ac9f905a-3855-4232-8256-59816f27354e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:26.678937 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1de6290280a48a8a2802a1c6c8113df', 'request_id': 'req-ac9f905a-3855-4232-8256-59816f27354e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:26.703627 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1de6290280a48a8a2802a1c6c8113df', 'request_id': 'req-ac9f905a-3855-4232-8256-59816f27354e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:26.718608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] The token cache is being invalidate because project c1de6290280a48a8a2802a1c6c8113df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:26.719012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:26.719770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:26.721711 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1409/2794] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:26 2022] DELETE /identity/v3/projects/c1de6290280a48a8a2802a1c6c8113df => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:26.725659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:26.737597 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075df4c5-a534-42c2-9c10-dc443689bf86 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:26.810929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:26.811833 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:26.812465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:26.812465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] PATH_INFO: `/v3/users/c7a77aa689b5424d9b4c5b8c7665b736` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:26.813693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a77aa689b5424d9b4c5b8c7665b736)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:26.815729 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:26.817510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:26.980580 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a77aa689b5424d9b4c5b8c7665b736'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:26.988016 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a77aa689b5424d9b4c5b8c7665b736'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.044997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a77aa689b5424d9b4c5b8c7665b736'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.045615 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a77aa689b5424d9b4c5b8c7665b736'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.057969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d7106ae2-2c82-4ec7-b600-3c1c21d3eea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a77aa689b5424d9b4c5b8c7665b736'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.064788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1386/2795] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:26 2022] DELETE /identity/v3/users/c7a77aa689b5424d9b4c5b8c7665b736 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:27.485465 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:27.487187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:27.500326 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f905a-3855-4232-8256-59816f27354e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:27.557876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:27.558514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:27.559279 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:27.559595 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] PATH_INFO: `/v3/projects/a0d2104094114f2dabc8d39dcee5bf28` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:27.560848 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d2104094114f2dabc8d39dcee5bf28)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:27.562744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:27.678439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d2104094114f2dabc8d39dcee5bf28', 'request_id': 'req-dde81daf-1bd2-4baa-8b18-5e948da527d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.752622 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d2104094114f2dabc8d39dcee5bf28', 'request_id': 'req-dde81daf-1bd2-4baa-8b18-5e948da527d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.759503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d2104094114f2dabc8d39dcee5bf28', 'request_id': 'req-dde81daf-1bd2-4baa-8b18-5e948da527d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.771757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] The token cache is being invalidate because project a0d2104094114f2dabc8d39dcee5bf28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:27.772053 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.772701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-dde81daf-1bd2-4baa-8b18-5e948da527d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:27.774253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1410/2796] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:27 2022] DELETE /identity/v3/projects/a0d2104094114f2dabc8d39dcee5bf28 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:27.779235 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0beb79b-4995-4dd5-ae4f-965112be9a1a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:27.779585 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0beb79b-4995-4dd5-ae4f-965112be9a1a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:27.779883 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0beb79b-4995-4dd5-ae4f-965112be9a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:27.800724 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e0beb79b-4995-4dd5-ae4f-965112be9a1a None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:27.801956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e0beb79b-4995-4dd5-ae4f-965112be9a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:27.863981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1387/2797] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:27.872113 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d56bc7-1399-4bde-98f5-59b118c25b70 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:27.872416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d56bc7-1399-4bde-98f5-59b118c25b70 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:27.872701 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d56bc7-1399-4bde-98f5-59b118c25b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:27.892430 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-62d56bc7-1399-4bde-98f5-59b118c25b70 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:27.893659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-62d56bc7-1399-4bde-98f5-59b118c25b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:27.901923 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1411/2798] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:27.906503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0beb79b-4995-4dd5-ae4f-965112be9a1a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:27.916460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0beb79b-4995-4dd5-ae4f-965112be9a1a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:27.922095 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:27.922844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:27.923154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:27.923496 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:27.924251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:14:27.924563 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:27.926166 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:27.938716 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1388/2799] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:14:27.943035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d56bc7-1399-4bde-98f5-59b118c25b70 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:27.952558 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d56bc7-1399-4bde-98f5-59b118c25b70 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:27.957520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:27.958144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:27.958447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:27.958726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:27.959396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:27.961543 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.040047 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1412/2800] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:27 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:28.045064 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.055057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913967-d415-4c1d-b000-bd0201fd5aa3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.112391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.113028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.113301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.113581 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.114324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.116336 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.128211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:28.192650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1389/2801] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:28 2022] POST /identity/v3/users => generated 284 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:14:28.196726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.207466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83de7b-defd-4839-afb7-cf769bd8db76 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.212216 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.212830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.213184 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.213427 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.214174 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.216687 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.226718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1413/2802] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:28.230662 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.241177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1ca19-3c1b-4163-bece-e6b9df2036a3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.245925 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654c4f1-3f58-45af-8a33-0e828f53794f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.246594 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654c4f1-3f58-45af-8a33-0e828f53794f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.246876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654c4f1-3f58-45af-8a33-0e828f53794f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.247148 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654c4f1-3f58-45af-8a33-0e828f53794f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.247900 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654c4f1-3f58-45af-8a33-0e828f53794f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.249521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654c4f1-3f58-45af-8a33-0e828f53794f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.261065 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1390/2803] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:28.265263 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.275204 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f89c56-97da-4e2e-807c-e377e84e7e91 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.280786 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.281504 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.281784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.282059 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] PATH_INFO: `/v3/projects/de2e8410d8ce4521a5ea5a06a7d0860d/users/40a8fe94a0044514a755dabf4a480738/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.297946 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2e8410d8ce4521a5ea5a06a7d0860d, user_id=40a8fe94a0044514a755dabf4a480738, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.300586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.303606 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:28.398534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1414/2804] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:28 2022] PUT /identity/v3/projects/de2e8410d8ce4521a5ea5a06a7d0860d/users/40a8fe94a0044514a755dabf4a480738/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:28.409485 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0e263-ff7f-4cc9-bf3b-9c1b99d2e0e4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.409777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0e263-ff7f-4cc9-bf3b-9c1b99d2e0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.410083 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0e263-ff7f-4cc9-bf3b-9c1b99d2e0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.438786 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9ba0e263-ff7f-4cc9-bf3b-9c1b99d2e0e4 None None] MFA Rules not processed for user `40a8fe94a0044514a755dabf4a480738`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:28.440054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ba0e263-ff7f-4cc9-bf3b-9c1b99d2e0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:28.505315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1391/2805] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:14:28.509789 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.571835 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b6353-d4a4-44f9-9b30-0610c5cb825b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.576999 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.577614 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.577898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.578173 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.578833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.580773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.662151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1415/2806] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:28 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:28.666434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0e263-ff7f-4cc9-bf3b-9c1b99d2e0e4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.677630 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0e263-ff7f-4cc9-bf3b-9c1b99d2e0e4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.733388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.734005 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.734292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.734571 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.735234 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.737180 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.816262 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1392/2807] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:28 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:28.820544 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.834380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fde00-7faf-4c72-a66e-c86d2c241ec3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.836557 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.837207 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.837487 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.837759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.838447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.840759 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.851273 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1416/2808] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:28.855334 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.865515 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08929807-bd15-41ec-9623-612dff378cb3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:28.870460 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:28.871128 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:28.871479 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:28.871847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] PATH_INFO: `/v3/projects/3bedde995ca3486483740d047e06fa14/users/dd676403e2764599a9fa50f622dbbaaf/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:28.888246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bedde995ca3486483740d047e06fa14, user_id=dd676403e2764599a9fa50f622dbbaaf, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:28.890877 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:28.975640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1393/2809] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:28 2022] PUT /identity/v3/projects/3bedde995ca3486483740d047e06fa14/users/dd676403e2764599a9fa50f622dbbaaf/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:28.979892 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:28.990956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24b5cc-6b61-4aca-b36d-72ad8bf87df0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:29.049503 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74476cb8-b83e-4936-981a-8a6f294af8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:29.050124 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74476cb8-b83e-4936-981a-8a6f294af8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:29.050408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74476cb8-b83e-4936-981a-8a6f294af8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:29.050670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74476cb8-b83e-4936-981a-8a6f294af8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:29.051363 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74476cb8-b83e-4936-981a-8a6f294af8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:29.052998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74476cb8-b83e-4936-981a-8a6f294af8b2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:29.066672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1417/2810] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:29.067822 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:29.078669 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e20e9-c3c7-45da-9f02-925cc7effd0a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:29.084310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:29.094174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:29.094174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:29.094174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] PATH_INFO: `/v3/projects/3bedde995ca3486483740d047e06fa14/users/dd676403e2764599a9fa50f622dbbaaf/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:29.094174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bedde995ca3486483740d047e06fa14, user_id=dd676403e2764599a9fa50f622dbbaaf, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:29.094174 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:29.101731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1394/2811] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:29 2022] PUT /identity/v3/projects/3bedde995ca3486483740d047e06fa14/users/dd676403e2764599a9fa50f622dbbaaf/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:29.113390 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:29.113684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:29.113968 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:29.144432 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] MFA Rules not processed for user `dd676403e2764599a9fa50f622dbbaaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:29.145725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:29.199760 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:29.217863 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1418/2812] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:29.229246 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:29.231953 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:29.262698 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:29.289478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e84aa-253d-4475-ad8e-1f50b5343280 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:29.294076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fadbd-fed3-4563-aed6-fef17cd71a81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:29.301191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fadbd-fed3-4563-aed6-fef17cd71a81 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:29.301191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fadbd-fed3-4563-aed6-fef17cd71a81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:29.301191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fadbd-fed3-4563-aed6-fef17cd71a81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:29.304293 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fadbd-fed3-4563-aed6-fef17cd71a81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:29.305737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fadbd-fed3-4563-aed6-fef17cd71a81 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:29.315129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1395/2813] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:29.430359 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:29.431768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:29.444176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a729-9c55-49b4-b63e-83a32ab43a63 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:29.447784 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:29.448396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:29.448677 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:29.448951 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:29.506306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:29.507846 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:29.520484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1419/2814] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:32.330893 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66efbd-6478-4bf0-bb80-3144507c9d71 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:32.331453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66efbd-6478-4bf0-bb80-3144507c9d71 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:32.331453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66efbd-6478-4bf0-bb80-3144507c9d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:32.351779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5e66efbd-6478-4bf0-bb80-3144507c9d71 None None] MFA Rules not processed for user `54a4d08f57a24222ac7f28c57cbe3c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:32.353952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5e66efbd-6478-4bf0-bb80-3144507c9d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:32.408781 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1396/2815] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:32.645522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:32.646970 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:32.705118 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec65c1-d2d7-46af-b9ca-0a23e3039f3b service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:32.708732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d56c57-e179-4837-925b-12d212077344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:32.709353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56c57-e179-4837-925b-12d212077344 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:32.709637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56c57-e179-4837-925b-12d212077344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:32.709915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56c57-e179-4837-925b-12d212077344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:32.711757 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e0d56c57-e179-4837-925b-12d212077344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:32.721393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d56c57-e179-4837-925b-12d212077344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:32.722851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d56c57-e179-4837-925b-12d212077344 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:32.732473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1420/2816] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:34.521897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66efbd-6478-4bf0-bb80-3144507c9d71 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:34.584916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66efbd-6478-4bf0-bb80-3144507c9d71 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:34.589639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:34.590402 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:34.590685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:34.591024 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] PATH_INFO: `/v3/users/40a8fe94a0044514a755dabf4a480738` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:34.592218 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a8fe94a0044514a755dabf4a480738)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:34.594137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:34.807107 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a8fe94a0044514a755dabf4a480738'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:34.814769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a8fe94a0044514a755dabf4a480738'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:34.889529 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a8fe94a0044514a755dabf4a480738'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:34.890434 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a8fe94a0044514a755dabf4a480738'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:34.901840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:34.904626 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a8fe94a0044514a755dabf4a480738'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:34.910687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1397/2817] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:34 2022] DELETE /identity/v3/users/40a8fe94a0044514a755dabf4a480738 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:34.915353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:34.915864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:34.915977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:34.936842 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:34.938080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:35.000159 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1421/2818] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:35.009843 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:35.011281 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:35.069126 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4b907-5686-41aa-8c99-24e44ddff38c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:35.072670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:35.073290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:35.073575 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:35.073851 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:35.083346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:35.085255 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:35.095868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1398/2819] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:35.568872 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:35.570369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:35.582419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57bbc-09ab-4ced-839b-50fecc3e77fd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:35.587063 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:35.587743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:35.588034 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:35.588315 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] PATH_INFO: `/v3/projects/de2e8410d8ce4521a5ea5a06a7d0860d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:35.589479 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2e8410d8ce4521a5ea5a06a7d0860d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:35.591799 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:35.680952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:35.696810 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2e8410d8ce4521a5ea5a06a7d0860d', 'request_id': 'req-152e5587-5b29-40e2-81e7-ab646a86d797'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:35.793715 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2e8410d8ce4521a5ea5a06a7d0860d', 'request_id': 'req-152e5587-5b29-40e2-81e7-ab646a86d797'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:35.800183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2e8410d8ce4521a5ea5a06a7d0860d', 'request_id': 'req-152e5587-5b29-40e2-81e7-ab646a86d797'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:35.812198 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] The token cache is being invalidate because project de2e8410d8ce4521a5ea5a06a7d0860d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:35.812494 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:35.813149 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:35.814699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1422/2820] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:35 2022] DELETE /identity/v3/projects/de2e8410d8ce4521a5ea5a06a7d0860d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:35.818735 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:35.829029 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9578e-6398-4eea-9b65-a7cc266c13a9 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:35.889297 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:35.890118 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:35.890403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:35.890688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] PATH_INFO: `/v3/users/dd676403e2764599a9fa50f622dbbaaf` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:35.891876 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd676403e2764599a9fa50f622dbbaaf)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:35.894080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:36.120712 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd676403e2764599a9fa50f622dbbaaf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:36.128432 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd676403e2764599a9fa50f622dbbaaf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:36.204080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd676403e2764599a9fa50f622dbbaaf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:36.204699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd676403e2764599a9fa50f622dbbaaf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:36.217993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-07eafd44-25cd-408a-b92b-ad9b4162df99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd676403e2764599a9fa50f622dbbaaf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:36.224296 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1399/2821] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:35 2022] DELETE /identity/v3/users/dd676403e2764599a9fa50f622dbbaaf => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:36.834458 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:36.836255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:36.851852 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e5587-5b29-40e2-81e7-ab646a86d797 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:36.925843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:36.926489 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:36.926770 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:36.927042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] PATH_INFO: `/v3/projects/3bedde995ca3486483740d047e06fa14` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:36.928224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bedde995ca3486483740d047e06fa14)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:36.930129 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:37.076608 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bedde995ca3486483740d047e06fa14', 'request_id': 'req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:37.158251 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bedde995ca3486483740d047e06fa14', 'request_id': 'req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:37.165313 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bedde995ca3486483740d047e06fa14', 'request_id': 'req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:37.177882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] The token cache is being invalidate because project 3bedde995ca3486483740d047e06fa14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:37.178172 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:37.178820 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b0b21dcc-ffbb-490c-b96a-1c543c0cacbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:37.180384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1423/2822] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:36 2022] DELETE /identity/v3/projects/3bedde995ca3486483740d047e06fa14 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:37.185951 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d0508-37d5-4451-ba38-aac717f11f44 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.186438 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d0508-37d5-4451-ba38-aac717f11f44 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.192523 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d0508-37d5-4451-ba38-aac717f11f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.206977 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5c1d0508-37d5-4451-ba38-aac717f11f44 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:37.208282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5c1d0508-37d5-4451-ba38-aac717f11f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:37.218513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:37.274274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1400/2823] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:37.282259 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fb692-3d6c-4ca5-b520-f46c50e57137 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.282554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fb692-3d6c-4ca5-b520-f46c50e57137 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.282869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fb692-3d6c-4ca5-b520-f46c50e57137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.305058 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bb1fb692-3d6c-4ca5-b520-f46c50e57137 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:37.306276 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bb1fb692-3d6c-4ca5-b520-f46c50e57137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:37.314894 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1424/2824] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:37.319417 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d0508-37d5-4451-ba38-aac717f11f44 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:37.330499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d0508-37d5-4451-ba38-aac717f11f44 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:37.335157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:37.335816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.336097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.336367 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.337092 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:14:37.337394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:37.339008 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:37.354456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1401/2825] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:14:37.358718 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fb692-3d6c-4ca5-b520-f46c50e57137 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:37.368181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fb692-3d6c-4ca5-b520-f46c50e57137 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:37.373477 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:37.374099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.374382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.374658 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.375329 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:37.377288 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:37.458282 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1425/2826] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:37 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:37.462521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:37.473127 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65218cb0-2d4b-4de0-8687-b6bfec3a7870 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:37.532014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:37.532639 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.532931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.533207 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.533888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:37.535873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:37.618481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1402/2827] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:37 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:37.619773 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:37.631726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d3ae8-5f74-467a-9bf5-2840b8a692f3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:37.636852 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:37.637590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.637873 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.638151 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.638911 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:37.655551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:37.655551 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1426/2828] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:37.658769 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:37.670157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ad4fe-5dfb-4adb-92cc-9e4460bf39c3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:37.674929 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf46a1-cc65-46ff-b63d-fed91be43897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:37.675579 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf46a1-cc65-46ff-b63d-fed91be43897 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.675873 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf46a1-cc65-46ff-b63d-fed91be43897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.676231 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf46a1-cc65-46ff-b63d-fed91be43897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.679323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf46a1-cc65-46ff-b63d-fed91be43897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:37.679323 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf46a1-cc65-46ff-b63d-fed91be43897 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:37.694922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1403/2829] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:37.697031 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:37.707500 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053339e-1770-4f0b-a6bb-60713c9ef513 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:37.712236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:37.713900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.713900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.713900 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] PATH_INFO: `/v3/projects/b070ea5c48da4358814c83f7a1f28268/users/7b0fdadca32045e5827e2bb851a47c96/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.715953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:37.729147 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b070ea5c48da4358814c83f7a1f28268, user_id=7b0fdadca32045e5827e2bb851a47c96, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:37.732459 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:37.799396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1427/2830] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:37 2022] PUT /identity/v3/projects/b070ea5c48da4358814c83f7a1f28268/users/7b0fdadca32045e5827e2bb851a47c96/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:37.810685 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e6aac-ec01-4611-87b2-c9f28bbe26dc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.810976 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e6aac-ec01-4611-87b2-c9f28bbe26dc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.811254 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e6aac-ec01-4611-87b2-c9f28bbe26dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.841044 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1b8e6aac-ec01-4611-87b2-c9f28bbe26dc None None] MFA Rules not processed for user `7b0fdadca32045e5827e2bb851a47c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:37.842449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1b8e6aac-ec01-4611-87b2-c9f28bbe26dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:37.907513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1404/2831] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:37.912018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:37.973692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57c0f-cd7d-4bcd-be7b-c0685e18800f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:37.978501 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:37.979123 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:37.979408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:37.979736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:37.981193 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:37.983111 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:38.062266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1428/2832] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:37 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:38.066449 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e6aac-ec01-4611-87b2-c9f28bbe26dc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:38.078001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e6aac-ec01-4611-87b2-c9f28bbe26dc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:38.086081 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:38.135650 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:38.136612 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.136992 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.137211 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:38.137878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:38.139834 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:38.216236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1405/2833] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:38 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:38.220402 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:38.231721 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ada4a-b3b9-41cc-805b-13632799a187 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:38.236482 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:38.237098 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.237382 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.237660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:38.238353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:38.240014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:38.253736 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1429/2834] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:14:38.257889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:38.268568 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18088f-37e5-43f8-b103-db852a45d1de admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:38.273198 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:38.273850 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.274129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.274403 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] PATH_INFO: `/v3/projects/0a7ef7c9f91f44e0803878c5d49ab56d/users/ed288b8b0a114e4cb7f96f53fefe7470/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:38.289408 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7ef7c9f91f44e0803878c5d49ab56d, user_id=ed288b8b0a114e4cb7f96f53fefe7470, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:38.292115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:38.363652 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1406/2835] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:38 2022] PUT /identity/v3/projects/0a7ef7c9f91f44e0803878c5d49ab56d/users/ed288b8b0a114e4cb7f96f53fefe7470/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:38.367651 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:38.377699 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf87008-06a9-4cb9-ad64-fc21dd58faf6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:38.432973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fb1be-7586-4991-82ad-41933c942364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:38.433692 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fb1be-7586-4991-82ad-41933c942364 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.433967 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fb1be-7586-4991-82ad-41933c942364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.434250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fb1be-7586-4991-82ad-41933c942364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:38.434939 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fb1be-7586-4991-82ad-41933c942364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:38.436586 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fb1be-7586-4991-82ad-41933c942364 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:38.445994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1430/2836] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:38.450817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:38.461015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56283ca-ea04-4882-bf3e-3c4f3ae28610 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:38.466237 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:38.466888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.467170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.467442 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] PATH_INFO: `/v3/projects/0a7ef7c9f91f44e0803878c5d49ab56d/users/ed288b8b0a114e4cb7f96f53fefe7470/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:38.469497 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7ef7c9f91f44e0803878c5d49ab56d, user_id=ed288b8b0a114e4cb7f96f53fefe7470, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:38.472179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:38.482384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1407/2837] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:38 2022] PUT /identity/v3/projects/0a7ef7c9f91f44e0803878c5d49ab56d/users/ed288b8b0a114e4cb7f96f53fefe7470/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:14:38.493536 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d624bf-440c-4ff8-a250-40f998f5db5f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.493832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d624bf-440c-4ff8-a250-40f998f5db5f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.494112 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d624bf-440c-4ff8-a250-40f998f5db5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:38.522210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a1d624bf-440c-4ff8-a250-40f998f5db5f None None] MFA Rules not processed for user `ed288b8b0a114e4cb7f96f53fefe7470`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:38.523419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a1d624bf-440c-4ff8-a250-40f998f5db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:38.564483 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:38.591484 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1431/2838] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:38.602772 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:38.604483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:38.663396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a810e-0eab-44b8-b0dc-dc42fe7a7ad5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:38.667036 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:38.667946 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.668366 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.668681 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:38.677537 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:38.678950 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:38.688888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1408/2839] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:38.932345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d624bf-440c-4ff8-a250-40f998f5db5f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:38.942915 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d624bf-440c-4ff8-a250-40f998f5db5f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:38.946733 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06255c1-88cb-4708-93f4-979e638abc1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:38.947349 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06255c1-88cb-4708-93f4-979e638abc1d service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:38.947670 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06255c1-88cb-4708-93f4-979e638abc1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:38.947952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06255c1-88cb-4708-93f4-979e638abc1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:39.008080 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06255c1-88cb-4708-93f4-979e638abc1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:39.009534 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06255c1-88cb-4708-93f4-979e638abc1d service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:39.019521 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1432/2840] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:45.179885 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:45.241965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f38e33-4589-4d84-87de-324ede1fddc5 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:45.246505 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:45.247175 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:45.247456 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:45.247762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] PATH_INFO: `/v3/users/7b0fdadca32045e5827e2bb851a47c96` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:45.248913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0fdadca32045e5827e2bb851a47c96)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:45.251503 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:45.256994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:45.399981 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0fdadca32045e5827e2bb851a47c96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:45.407799 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0fdadca32045e5827e2bb851a47c96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:45.466001 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0fdadca32045e5827e2bb851a47c96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:45.466619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0fdadca32045e5827e2bb851a47c96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:45.479878 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0fdadca32045e5827e2bb851a47c96'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:45.486584 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1409/2841] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:45 2022] DELETE /identity/v3/users/7b0fdadca32045e5827e2bb851a47c96 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:45.491131 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:45.491636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:45.491777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:45.510428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:45.513228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:45.570824 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1433/2842] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:45.580809 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:45.582219 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:45.639715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaa420-290d-408b-a508-509d7e04b2de admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:45.644723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:45.645349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:45.645634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:45.645911 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:45.655137 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:45.657134 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:45.666670 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1410/2843] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:46.087908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:46.089383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:46.101638 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43600781-ac5b-4a6e-ad10-10df26959934 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:46.106414 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:46.107068 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:46.107353 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:46.107660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] PATH_INFO: `/v3/projects/b070ea5c48da4358814c83f7a1f28268` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:46.108869 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b070ea5c48da4358814c83f7a1f28268)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:46.110898 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:46.199665 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b070ea5c48da4358814c83f7a1f28268', 'request_id': 'req-df7469af-2d33-48ae-97b1-f8f37ad0fb64'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.254845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b070ea5c48da4358814c83f7a1f28268', 'request_id': 'req-df7469af-2d33-48ae-97b1-f8f37ad0fb64'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.261103 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b070ea5c48da4358814c83f7a1f28268', 'request_id': 'req-df7469af-2d33-48ae-97b1-f8f37ad0fb64'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.269929 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:46.273731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] The token cache is being invalidate because project b070ea5c48da4358814c83f7a1f28268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:46.274023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.274661 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.276210 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1434/2844] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:46 2022] DELETE /identity/v3/projects/b070ea5c48da4358814c83f7a1f28268 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:46.280480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:46.290394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc85a87-1453-480e-9e89-c1cee3d66b6a service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:46.347283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:46.348080 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:46.348373 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:46.348679 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] PATH_INFO: `/v3/users/ed288b8b0a114e4cb7f96f53fefe7470` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:46.349899 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed288b8b0a114e4cb7f96f53fefe7470)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:46.351941 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:46.530177 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:46.544023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed288b8b0a114e4cb7f96f53fefe7470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.551727 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed288b8b0a114e4cb7f96f53fefe7470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.654723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed288b8b0a114e4cb7f96f53fefe7470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.655343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed288b8b0a114e4cb7f96f53fefe7470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.668271 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b0c7a18d-9c24-427b-bc2d-522c82a752ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed288b8b0a114e4cb7f96f53fefe7470'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:46.675343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1411/2845] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:14:46 2022] DELETE /identity/v3/users/ed288b8b0a114e4cb7f96f53fefe7470 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:47.119685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:47.123947 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:47.138447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7469af-2d33-48ae-97b1-f8f37ad0fb64 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:47.209292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:47.209943 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.210228 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.210507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] PATH_INFO: `/v3/projects/0a7ef7c9f91f44e0803878c5d49ab56d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.211768 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7ef7c9f91f44e0803878c5d49ab56d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:47.213676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:47.342572 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7ef7c9f91f44e0803878c5d49ab56d', 'request_id': 'req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:47.415646 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7ef7c9f91f44e0803878c5d49ab56d', 'request_id': 'req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:47.424408 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7ef7c9f91f44e0803878c5d49ab56d', 'request_id': 'req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:47.437126 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] The token cache is being invalidate because project 0a7ef7c9f91f44e0803878c5d49ab56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:14:47.437417 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:47.438237 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c9bd8b8e-d362-4e1a-9ea3-0def8ebca4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:14:47.440466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1435/2846] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:14:47 2022] DELETE /identity/v3/projects/0a7ef7c9f91f44e0803878c5d49ab56d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:47.446318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09fab-06ef-4412-a1f3-f0fe51fdfff4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.446844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09fab-06ef-4412-a1f3-f0fe51fdfff4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.446844 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09fab-06ef-4412-a1f3-f0fe51fdfff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.467191 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-59e09fab-06ef-4412-a1f3-f0fe51fdfff4 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:47.468450 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-59e09fab-06ef-4412-a1f3-f0fe51fdfff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:47.528748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1412/2847] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:47.537310 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34d276-cefc-48c4-acc6-c9cfbe0972c6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.537605 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34d276-cefc-48c4-acc6-c9cfbe0972c6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.537885 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34d276-cefc-48c4-acc6-c9cfbe0972c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.559372 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5b34d276-cefc-48c4-acc6-c9cfbe0972c6 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:47.560676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5b34d276-cefc-48c4-acc6-c9cfbe0972c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:47.568952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1436/2848] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:47.573270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e09fab-06ef-4412-a1f3-f0fe51fdfff4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:47.583007 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e09fab-06ef-4412-a1f3-f0fe51fdfff4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:47.587645 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:47.588253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.588540 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.588812 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.589500 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:14:47.589803 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:47.591395 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:47.603236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1413/2849] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:14:47.607767 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34d276-cefc-48c4-acc6-c9cfbe0972c6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:47.617849 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34d276-cefc-48c4-acc6-c9cfbe0972c6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:47.622577 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:47.623236 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.623472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.623783 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.624447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:47.626466 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:47.708132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1437/2850] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:47 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:47.712521 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:47.723553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4caf9-6133-4de5-a4c1-c22e5f1fdcd8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:47.783196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:47.783868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.784155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.784430 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.785157 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:47.787125 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:47.878113 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1414/2851] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:47 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:14:47.881966 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:47.892908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1383-a620-423a-9c6c-c449b804c4ec admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:47.897637 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:47.898253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.898564 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.898843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.899607 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:47.901240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:47.911866 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1438/2852] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:47.916087 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:47.927170 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2fa8c-9778-4e26-9b5e-879b021e4cbc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:47.931868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68eba2-5d02-49e8-85c4-ee4acb6530c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:47.932476 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68eba2-5d02-49e8-85c4-ee4acb6530c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.932763 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68eba2-5d02-49e8-85c4-ee4acb6530c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.933042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68eba2-5d02-49e8-85c4-ee4acb6530c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.933740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68eba2-5d02-49e8-85c4-ee4acb6530c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:47.935370 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68eba2-5d02-49e8-85c4-ee4acb6530c2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:47.945964 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1415/2853] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:47.950088 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:47.960765 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977fd4-2dbf-4b66-ad7a-449d2125fb2f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:47.965555 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:47.969625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:47.969625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:47.969625 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] PATH_INFO: `/v3/projects/2ec3fa08bfc743988ae222f40deeb320/users/4441e116740b4e03b4e2a3912046db07/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:47.983376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec3fa08bfc743988ae222f40deeb320, user_id=4441e116740b4e03b4e2a3912046db07, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:47.986553 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:48.068012 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1439/2854] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:47 2022] PUT /identity/v3/projects/2ec3fa08bfc743988ae222f40deeb320/users/4441e116740b4e03b4e2a3912046db07/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:48.080659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aeb12b-cf34-45b5-a5b6-29b82b4d00dc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.080954 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aeb12b-cf34-45b5-a5b6-29b82b4d00dc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.081261 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aeb12b-cf34-45b5-a5b6-29b82b4d00dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.106664 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:48.109945 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f4aeb12b-cf34-45b5-a5b6-29b82b4d00dc None None] MFA Rules not processed for user `4441e116740b4e03b4e2a3912046db07`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:48.112898 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f4aeb12b-cf34-45b5-a5b6-29b82b4d00dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:48.180057 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1416/2855] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:48.186740 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:48.215345 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:48.249258 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d60997-8f7e-485b-b028-6139e5ea7733 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:48.255761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:48.255761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.255761 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.256139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.256580 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:48.258529 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:48.342688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1440/2856] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:14:48.347068 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aeb12b-cf34-45b5-a5b6-29b82b4d00dc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:48.358260 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aeb12b-cf34-45b5-a5b6-29b82b4d00dc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:48.420002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:48.420536 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.420825 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.421105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.421775 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:48.423731 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:48.518966 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1417/2857] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:14:48 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:14:48.522908 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:48.533868 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c535da-0666-46b2-8a82-e8ab918befcf admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:48.538569 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:48.539181 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.539497 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.539806 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.540505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:48.542366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:48.552384 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1441/2858] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:14:48.556994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:48.568023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e570692-8e09-45d8-a2a1-dcdffbbadf5c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:48.573014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:48.573737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.574028 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.574315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] PATH_INFO: `/v3/projects/cca2020e4fad4cb4ab97ee93d8e0e771/users/4e2bdd69e04e40f2bff268b8c398cdf3/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.589829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca2020e4fad4cb4ab97ee93d8e0e771, user_id=4e2bdd69e04e40f2bff268b8c398cdf3, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:48.592619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:48.636988 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1418/2859] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:48 2022] PUT /identity/v3/projects/cca2020e4fad4cb4ab97ee93d8e0e771/users/4e2bdd69e04e40f2bff268b8c398cdf3/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:48.641615 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:48.651596 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227dac3-f37f-4d49-91da-563016db50ea admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:48.708498 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00473a49-c3c7-460b-996b-dea405809efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:48.709035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473a49-c3c7-460b-996b-dea405809efb admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.709321 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473a49-c3c7-460b-996b-dea405809efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.709641 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473a49-c3c7-460b-996b-dea405809efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.710376 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00473a49-c3c7-460b-996b-dea405809efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:48.712026 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00473a49-c3c7-460b-996b-dea405809efb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:48.722253 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1442/2860] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 08:14:48.726599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:48.737426 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435f002-8df2-4c54-8fab-6e78fd225e46 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:48.742074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:48.742788 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.743037 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.743343 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] PATH_INFO: `/v3/projects/cca2020e4fad4cb4ab97ee93d8e0e771/users/4e2bdd69e04e40f2bff268b8c398cdf3/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.745383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca2020e4fad4cb4ab97ee93d8e0e771, user_id=4e2bdd69e04e40f2bff268b8c398cdf3, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:48.748042 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:48.759936 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1419/2861] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:14:48 2022] PUT /identity/v3/projects/cca2020e4fad4cb4ab97ee93d8e0e771/users/4e2bdd69e04e40f2bff268b8c398cdf3/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:14:48.774451 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.774744 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.775023 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.803195 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] MFA Rules not processed for user `4e2bdd69e04e40f2bff268b8c398cdf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:14:48.807419 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:48.871762 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1443/2862] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:14:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:14:48.883140 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:48.884868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:48.944446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34581362-3afb-4c21-a055-603690d56d61 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:48.948129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:48.948755 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:48.949040 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:48.949319 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:48.957792 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:14:49.006251 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:49.007686 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:49.016586 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1420/2863] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:54.150278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:14:54.151877 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:14:54.164225 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1acdc-7aa8-43de-982f-103bbcd0870c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:14:54.168833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242c9c2-a02b-4490-bbb1-abc1cca969cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317df4b0baa54808af9b7918ff5c3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:14:54.168833 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242c9c2-a02b-4490-bbb1-abc1cca969cd service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:14:54.169056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242c9c2-a02b-4490-bbb1-abc1cca969cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:14:54.169322 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242c9c2-a02b-4490-bbb1-abc1cca969cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:14:54.179045 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242c9c2-a02b-4490-bbb1-abc1cca969cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:14:54.180635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242c9c2-a02b-4490-bbb1-abc1cca969cd service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:14:54.190881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1444/2864] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:14:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:14:59.066315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=23 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:04.185818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:08.503906 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:08.569997 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b4ee2-9998-42da-8ac1-fefce6bdbd11 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:08.574748 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:08.575396 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:08.575715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:08.576027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] PATH_INFO: `/v3/users/4441e116740b4e03b4e2a3912046db07` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:08.577230 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4441e116740b4e03b4e2a3912046db07)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:08.579147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:08.730820 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4441e116740b4e03b4e2a3912046db07'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:08.737952 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4441e116740b4e03b4e2a3912046db07'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:08.814384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4441e116740b4e03b4e2a3912046db07'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:08.815014 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4441e116740b4e03b4e2a3912046db07'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:08.828398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4441e116740b4e03b4e2a3912046db07'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:08.834374 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1421/2865] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:15:08 2022] DELETE /identity/v3/users/4441e116740b4e03b4e2a3912046db07 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:15:08.839211 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:08.839800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:08.839800 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:08.858725 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:15:08.859962 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:08.921141 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1445/2866] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:15:08.931622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:08.933441 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:08.992869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef3a9-69d8-4d72-9ede-8192e5129843 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:08.997027 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:08.997699 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:08.997999 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:08.998277 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:09.008858 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:09.010348 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:09.020146 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1422/2867] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:15:09.450973 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:09.452601 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:09.466108 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa999b7-781c-4130-8149-fe71407a37ca None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:09.471219 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:09.471764 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:09.472056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:09.472336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] PATH_INFO: `/v3/projects/2ec3fa08bfc743988ae222f40deeb320` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:09.473574 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec3fa08bfc743988ae222f40deeb320)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:09.475684 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:09.592436 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec3fa08bfc743988ae222f40deeb320', 'request_id': 'req-fca1ae50-386e-4375-b940-f52c1c1b82d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:09.648862 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec3fa08bfc743988ae222f40deeb320', 'request_id': 'req-fca1ae50-386e-4375-b940-f52c1c1b82d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:09.654603 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec3fa08bfc743988ae222f40deeb320', 'request_id': 'req-fca1ae50-386e-4375-b940-f52c1c1b82d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:09.666209 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] The token cache is being invalidate because project 2ec3fa08bfc743988ae222f40deeb320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:15:09.666758 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:09.667397 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:09.669035 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1446/2868] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:15:09 2022] DELETE /identity/v3/projects/2ec3fa08bfc743988ae222f40deeb320 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:15:09.673466 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:09.683480 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b325de-af50-4fa2-b776-8229fb7bb59e service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:09.701569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:09.740372 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:09.741101 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:09.741756 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:09.742100 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] PATH_INFO: `/v3/users/4e2bdd69e04e40f2bff268b8c398cdf3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:09.743290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2bdd69e04e40f2bff268b8c398cdf3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:09.745259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:09.885203 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2bdd69e04e40f2bff268b8c398cdf3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:09.892074 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2bdd69e04e40f2bff268b8c398cdf3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.031105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2bdd69e04e40f2bff268b8c398cdf3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.031601 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2bdd69e04e40f2bff268b8c398cdf3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.045916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2bdd69e04e40f2bff268b8c398cdf3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.052321 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1423/2869] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:15:09 2022] DELETE /identity/v3/users/4e2bdd69e04e40f2bff268b8c398cdf3 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:15:10.434854 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:10.436809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:10.454590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1ae50-386e-4375-b940-f52c1c1b82d5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:10.517741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:10.518395 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:10.518676 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:10.518953 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] PATH_INFO: `/v3/projects/cca2020e4fad4cb4ab97ee93d8e0e771` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:10.520125 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca2020e4fad4cb4ab97ee93d8e0e771)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:10.522377 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:10.621809 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:10.638380 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca2020e4fad4cb4ab97ee93d8e0e771', 'request_id': 'req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.721817 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca2020e4fad4cb4ab97ee93d8e0e771', 'request_id': 'req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.727575 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca2020e4fad4cb4ab97ee93d8e0e771', 'request_id': 'req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.739650 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] The token cache is being invalidate because project cca2020e4fad4cb4ab97ee93d8e0e771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:15:10.739948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.740611 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:10.742163 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1447/2870] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:15:10 2022] DELETE /identity/v3/projects/cca2020e4fad4cb4ab97ee93d8e0e771 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:15:20.048528 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:20.779728 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:43.940762 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:43.942248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:43.957499 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950a91d-68e8-40ca-9152-739039fbef38 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:44.018143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:44.018800 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:44.019085 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:44.019359 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] PATH_INFO: `/v3/users/54a4d08f57a24222ac7f28c57cbe3c64` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:44.020556 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a4d08f57a24222ac7f28c57cbe3c64)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:44.022509 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:44.163894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a4d08f57a24222ac7f28c57cbe3c64'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.171184 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a4d08f57a24222ac7f28c57cbe3c64'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.225116 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a4d08f57a24222ac7f28c57cbe3c64'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.225684 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a4d08f57a24222ac7f28c57cbe3c64'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.238888 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c9ab0aad-e31d-4cd2-b888-5b7be825d489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a4d08f57a24222ac7f28c57cbe3c64'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.245793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1424/2871] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:15:43 2022] DELETE /identity/v3/users/54a4d08f57a24222ac7f28c57cbe3c64 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:15:44.500571 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:44.502336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:44.524541 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67032d3a-fd3b-4dfe-b071-46a6e4c1ba05 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:44.582328 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:44.582994 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:44.583280 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:44.583581 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] PATH_INFO: `/v3/projects/5e81732f21c54f07bbd838396f6cb0d6` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:44.584731 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e81732f21c54f07bbd838396f6cb0d6)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:44.586643 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:44.700245 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e81732f21c54f07bbd838396f6cb0d6', 'request_id': 'req-ed4858bf-3d22-4a85-8f63-e33c027b85a7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.779106 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e81732f21c54f07bbd838396f6cb0d6', 'request_id': 'req-ed4858bf-3d22-4a85-8f63-e33c027b85a7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.785319 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e81732f21c54f07bbd838396f6cb0d6', 'request_id': 'req-ed4858bf-3d22-4a85-8f63-e33c027b85a7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.796732 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] The token cache is being invalidate because project 5e81732f21c54f07bbd838396f6cb0d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:15:44.797022 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.797657 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ed4858bf-3d22-4a85-8f63-e33c027b85a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:15:44.799170 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1448/2872] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:15:44 2022] DELETE /identity/v3/projects/5e81732f21c54f07bbd838396f6cb0d6 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:15:44.804185 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f85960-e5bb-408b-88fb-0447ac0d432e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:44.804478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f85960-e5bb-408b-88fb-0447ac0d432e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:44.804757 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f85960-e5bb-408b-88fb-0447ac0d432e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:44.825158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-29f85960-e5bb-408b-88fb-0447ac0d432e None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:15:44.826358 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-29f85960-e5bb-408b-88fb-0447ac0d432e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:44.883173 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1425/2873] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:15:44.892374 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eacb5a-8ec7-489d-8105-17b987523fb7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:44.892669 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eacb5a-8ec7-489d-8105-17b987523fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:44.892952 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eacb5a-8ec7-489d-8105-17b987523fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:44.912378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c3eacb5a-8ec7-489d-8105-17b987523fb7 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:15:44.913589 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c3eacb5a-8ec7-489d-8105-17b987523fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:44.921845 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1449/2874] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:15:44.925970 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f85960-e5bb-408b-88fb-0447ac0d432e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:44.936886 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f85960-e5bb-408b-88fb-0447ac0d432e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:44.941453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:44.942063 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:44.942346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:44.942623 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:44.943318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 08:15:44.943659 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:44.945253 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:44.957478 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1426/2875] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 08:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 08:15:44.961635 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eacb5a-8ec7-489d-8105-17b987523fb7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:44.971378 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eacb5a-8ec7-489d-8105-17b987523fb7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:44.976215 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:44.976827 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:44.977116 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:44.977396 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:44.978073 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:44.980639 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:45.099685 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1450/2876] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:15:44 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:15:45.104435 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:45.114817 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282daf4-46a6-475f-acca-38c44997e058 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:45.124983 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:45.173660 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:45.174279 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:45.174569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:45.174852 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:45.175513 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:45.177461 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:45.270758 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1427/2877] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:15:45 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 08:15:45.275114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:45.285864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cdd73-6f25-4c83-af9a-a2913cc3b232 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:45.290948 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:45.292883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:45.292883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:45.292883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:45.293626 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:45.295231 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:45.305393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1451/2878] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:15:45.310189 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:45.320283 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d079f51-4f6b-4055-888c-c451cfc6bdf4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:45.325217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a4341-da6d-42bc-9591-b5605f52b2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:45.325831 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a4341-da6d-42bc-9591-b5605f52b2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:45.326143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a4341-da6d-42bc-9591-b5605f52b2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:45.326422 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a4341-da6d-42bc-9591-b5605f52b2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:45.327120 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a4341-da6d-42bc-9591-b5605f52b2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:45.328782 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a4341-da6d-42bc-9591-b5605f52b2b1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:45.339533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1428/2879] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:15:45.343435 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:45.353462 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ab53c-83bc-4658-858b-8a5ed9885050 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:45.358403 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:45.359183 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:45.359464 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:45.359769 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] PATH_INFO: `/v3/projects/73e81c6e477a41ddb15e47b0c486b33e/users/7d9d2555e71d4459b4fb907b764c1e79/roles/338a254b1f394ba2b6d9f5fb3885cc4d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:45.374956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e81c6e477a41ddb15e47b0c486b33e, user_id=7d9d2555e71d4459b4fb907b764c1e79, role_id=338a254b1f394ba2b6d9f5fb3885cc4d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:45.377621 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:45.436343 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1452/2880] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:15:45 2022] PUT /identity/v3/projects/73e81c6e477a41ddb15e47b0c486b33e/users/7d9d2555e71d4459b4fb907b764c1e79/roles/338a254b1f394ba2b6d9f5fb3885cc4d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:15:45.441390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bae958-820d-4ba9-ac36-dea7817ca583 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:45.441897 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bae958-820d-4ba9-ac36-dea7817ca583 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:45.442150 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bae958-820d-4ba9-ac36-dea7817ca583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:45.461176 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-90bae958-820d-4ba9-ac36-dea7817ca583 None None] MFA Rules not processed for user `3ab27936a00c428f91a5c67560723447`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:15:45.462391 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-90bae958-820d-4ba9-ac36-dea7817ca583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:45.522155 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1429/2881] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:15:45.532522 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:45.534255 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:45.565828 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:45.592064 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a777c2-20b7-4be6-9868-627fa426e2e9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:45.595864 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:45.596672 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:45.597266 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:45.597582 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:45.607815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:45.608009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:45.617136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1453/2882] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:15:50.813318 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e73651-6dfa-4f34-b1bd-d9de6a2fb72b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:50.813829 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e73651-6dfa-4f34-b1bd-d9de6a2fb72b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:50.813937 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e73651-6dfa-4f34-b1bd-d9de6a2fb72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:50.845384 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-58e73651-6dfa-4f34-b1bd-d9de6a2fb72b None None] MFA Rules not processed for user `7d9d2555e71d4459b4fb907b764c1e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:15:50.846633 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-58e73651-6dfa-4f34-b1bd-d9de6a2fb72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:50.922777 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1430/2883] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:15:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:15:50.927292 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:50.937729 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f60452-a2b0-475f-8dcd-8c00ecaa5e16 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:50.942271 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:50.942876 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:50.943158 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:50.943433 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:50.944139 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:50.946042 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:51.026739 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1454/2884] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 08:15:50 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:15:51.030959 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e73651-6dfa-4f34-b1bd-d9de6a2fb72b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:51.042596 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e73651-6dfa-4f34-b1bd-d9de6a2fb72b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:51.053840 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:51.102053 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:51.102682 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:51.102968 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:51.103287 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:51.104554 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:51.106473 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:51.202455 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1431/2885] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 08:15:51 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 08:15:51.206456 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:51.217488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac5829-2c10-42d7-928a-f70b298bd865 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:51.222226 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:51.222838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:51.223127 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:51.223439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:51.224217 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:51.225821 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:51.237362 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1455/2886] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:15:51.241431 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:51.252428 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904353b3-6ef1-4ccd-b281-300b59d99c1a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:51.257055 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:51.257715 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:51.258002 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:51.258282 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] PATH_INFO: `/v3/projects/59d326c393cf4ce9b78c854f112f1b30/users/00dd387613e0488cb4f9aabf9b0a409a/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:51.272956 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d326c393cf4ce9b78c854f112f1b30, user_id=00dd387613e0488cb4f9aabf9b0a409a, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:51.275816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:51.361948 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1432/2887] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:15:51 2022] PUT /identity/v3/projects/59d326c393cf4ce9b78c854f112f1b30/users/00dd387613e0488cb4f9aabf9b0a409a/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 08:15:51.366055 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:51.376985 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ee4e-1c80-4275-8762-20aa9c841ced admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:51.440882 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3393674-7c05-4d5e-85f7-c8d18d631c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:51.441633 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3393674-7c05-4d5e-85f7-c8d18d631c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:51.441998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3393674-7c05-4d5e-85f7-c8d18d631c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:51.442338 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3393674-7c05-4d5e-85f7-c8d18d631c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:51.443109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3393674-7c05-4d5e-85f7-c8d18d631c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:51.444856 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3393674-7c05-4d5e-85f7-c8d18d631c2c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:51.455838 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1456/2888] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 08:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 08:15:51.460931 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:51.472543 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8e50c-e398-4ee9-99e8-156f05ec8506 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:51.478159 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:51.479453 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:51.479889 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:51.480248 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] PATH_INFO: `/v3/projects/59d326c393cf4ce9b78c854f112f1b30/users/00dd387613e0488cb4f9aabf9b0a409a/roles/05421dac98a848f68d13ca21036dbfbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:51.482516 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d326c393cf4ce9b78c854f112f1b30, user_id=00dd387613e0488cb4f9aabf9b0a409a, role_id=05421dac98a848f68d13ca21036dbfbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:51.485259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:51.502167 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1433/2889] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 08:15:51 2022] PUT /identity/v3/projects/59d326c393cf4ce9b78c854f112f1b30/users/00dd387613e0488cb4f9aabf9b0a409a/roles/05421dac98a848f68d13ca21036dbfbb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:15:56.774590 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6f55-2d18-48be-a640-2c6b2d022aa5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:56.775609 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6f55-2d18-48be-a640-2c6b2d022aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:56.775822 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6f55-2d18-48be-a640-2c6b2d022aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:56.808549 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-436e6f55-2d18-48be-a640-2c6b2d022aa5 None None] MFA Rules not processed for user `00dd387613e0488cb4f9aabf9b0a409a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:15:56.810075 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-436e6f55-2d18-48be-a640-2c6b2d022aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:56.884860 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1457/2890] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:15:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:15:56.897215 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:56.898984 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:56.961383 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff04a2-f883-4545-a639-39f623efd8a4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:56.965774 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:56.966405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:56.966687 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:56.966960 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:57.029131 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:57.030634 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:57.040502 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1434/2891] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:15:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:15:57.372342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86420d84-9967-4648-b7d7-be0972315b8a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:57.372636 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86420d84-9967-4648-b7d7-be0972315b8a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:57.372916 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86420d84-9967-4648-b7d7-be0972315b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:57.392428 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-86420d84-9967-4648-b7d7-be0972315b8a None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:15:57.393654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-86420d84-9967-4648-b7d7-be0972315b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:57.407056 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:57.461934 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1458/2892] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:15:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:15:57.607483 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:15:57.609076 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:15:57.639969 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:15:57.686143 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f0e0f-6e16-4b7d-9d2d-d4eb8bd0c5e2 service glance] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:15:57.691315 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:15:57.692259 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:15:57.692446 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:15:57.692722 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:15:57.703397 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:15:57.705058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:15:57.714524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1435/2893] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:15:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:16:07.487818 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=17 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:07.711217 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=22 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:08.098294 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c93556-65b6-42c7-8009-8152e30fdb94 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:08.098746 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c93556-65b6-42c7-8009-8152e30fdb94 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:08.099030 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c93556-65b6-42c7-8009-8152e30fdb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:08.118794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a4c93556-65b6-42c7-8009-8152e30fdb94 None None] MFA Rules not processed for user `7d9d2555e71d4459b4fb907b764c1e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:16:08.120018 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a4c93556-65b6-42c7-8009-8152e30fdb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:08.126883 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1459/2894] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 08:16:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:16:08.165972 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:08.175376 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba36cc8-6c4a-4882-834b-43642ffb5fdc service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:08.179270 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565d9f8-186c-49fc-9280-e3681386e0d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f2160adab864eb9970b7767eeeb4789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:08.184569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565d9f8-186c-49fc-9280-e3681386e0d9 service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:08.184569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565d9f8-186c-49fc-9280-e3681386e0d9 service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:08.184569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565d9f8-186c-49fc-9280-e3681386e0d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:08.184569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c565d9f8-186c-49fc-9280-e3681386e0d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:08.254534 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565d9f8-186c-49fc-9280-e3681386e0d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:08.256023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565d9f8-186c-49fc-9280-e3681386e0d9 service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:08.268058 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1436/2895] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:16:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:16:08.880688 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c93556-65b6-42c7-8009-8152e30fdb94 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:08.938472 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c93556-65b6-42c7-8009-8152e30fdb94 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:08.942212 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:08.942881 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:08.943162 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:08.943439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:08.945232 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:08.960077 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:08.961523 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:08.973884 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1460/2896] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:16:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:16:15.575744 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3d5c-8940-443c-8938-488647c90921 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:15.576324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3d5c-8940-443c-8938-488647c90921 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:15.576324 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3d5c-8940-443c-8938-488647c90921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:15.596868 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a7ad3d5c-8940-443c-8938-488647c90921 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:16:15.598530 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a7ad3d5c-8940-443c-8938-488647c90921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:15.606352 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1437/2897] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 08:16:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 08:16:15.618979 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:15.631306 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb15276-2f1a-4f82-9951-20cb247467ed service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:15.636366 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:15.636992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:15.637278 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:15.637554 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:15.646520 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:15.648222 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:15.658027 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1461/2898] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:16:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:16:25.591210 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=37 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:25.654334 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=35 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:26.350082 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad3d5c-8940-443c-8938-488647c90921 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:26.356278 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad3d5c-8940-443c-8938-488647c90921 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:26.360169 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:26.360789 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:26.361070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:26.361394 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:26.363154 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:26.374045 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:26.375591 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:26.384190 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1438/2899] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:16:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:16:33.224144 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:33.225735 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:33.246109 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ad4c-f40c-4384-8d8d-316442460002 service swift] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:33.248597 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:33.249361 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:33.249743 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:33.250114 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:33.252342 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:33.265383 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:33.269187 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:33.278148 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1462/2900] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:16:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 08:16:36.380310 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:37.381425 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:37.391097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759a982-19ba-4e01-95cb-682ddebb86a5 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:37.394846 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:37.395458 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:37.395779 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:37.396054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:37.397798 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:37.474097 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:37.475767 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:37.487794 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1439/2901] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:16:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:16:40.802252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:40.803748 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:40.874389 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3baf62-5c93-4b0d-b6b8-5399cca06a05 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:40.879652 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:40.881260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:40.881260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:40.881260 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:40.882667 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:40.895016 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:40.896675 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:40.909617 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1463/2902] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:16:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:16:44.498847 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:44.500350 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:44.518965 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd15e71-85c5-4e20-baba-05f8b696fce2 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:44.523720 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:44.524368 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:44.524792 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:44.525054 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] PATH_INFO: `/v3/users/1bc01a0e20fd411b9df5d2f077784f33` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:44.526734 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc01a0e20fd411b9df5d2f077784f33)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:44.528761 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:44.697678 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc01a0e20fd411b9df5d2f077784f33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:44.704620 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc01a0e20fd411b9df5d2f077784f33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:44.764994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc01a0e20fd411b9df5d2f077784f33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:44.765619 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc01a0e20fd411b9df5d2f077784f33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:44.779329 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc01a0e20fd411b9df5d2f077784f33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:44.785390 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1440/2903] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:16:44 2022] DELETE /identity/v3/users/1bc01a0e20fd411b9df5d2f077784f33 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:16:45.019566 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:45.031919 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479fffd-4928-4ab6-824c-e277912eda1c service placement] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:45.091853 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:45.092505 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:45.092794 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:45.093076 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] PATH_INFO: `/v3/projects/eb938950a17942b9b4b1ca56f319e754` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:45.094240 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb938950a17942b9b4b1ca56f319e754)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:45.096264 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:45.176843 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb938950a17942b9b4b1ca56f319e754', 'request_id': 'req-c8a31d25-acda-443c-bafe-37eb19f4794e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.239178 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb938950a17942b9b4b1ca56f319e754', 'request_id': 'req-c8a31d25-acda-443c-bafe-37eb19f4794e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.245089 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb938950a17942b9b4b1ca56f319e754', 'request_id': 'req-c8a31d25-acda-443c-bafe-37eb19f4794e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.256317 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] The token cache is being invalidate because project eb938950a17942b9b4b1ca56f319e754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:16:45.256660 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.257295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.258830 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1464/2904] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:16:45 2022] DELETE /identity/v3/projects/eb938950a17942b9b4b1ca56f319e754 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:16:45.263726 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:45.265069 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:45.278916 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261fa0c-8a4f-458f-a811-9a8c4937e2fd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:45.335510 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:45.336179 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:45.336465 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:45.336743 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] PATH_INFO: `/v3/users/aa8f7ec2f17b407faa2180ea8070549f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:45.337913 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8f7ec2f17b407faa2180ea8070549f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:45.339869 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:45.345286 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:45.533994 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8f7ec2f17b407faa2180ea8070549f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.542979 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8f7ec2f17b407faa2180ea8070549f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.617658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8f7ec2f17b407faa2180ea8070549f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.618327 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8f7ec2f17b407faa2180ea8070549f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.631644 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8f7ec2f17b407faa2180ea8070549f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:45.638550 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1441/2905] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:16:45 2022] DELETE /identity/v3/users/aa8f7ec2f17b407faa2180ea8070549f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:16:45.888507 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:45.889921 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:45.905447 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31d25-acda-443c-bafe-37eb19f4794e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:45.909439 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:45.968423 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:45.969095 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:45.969379 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:45.969659 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] PATH_INFO: `/v3/projects/b7213e8868bc4d9086c873a6fe0d49ab` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:45.970813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7213e8868bc4d9086c873a6fe0d49ab)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:45.972956 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:46.097815 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7213e8868bc4d9086c873a6fe0d49ab', 'request_id': 'req-042c2003-e8cd-47e9-a7ae-947c79b91c34'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:46.166247 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7213e8868bc4d9086c873a6fe0d49ab', 'request_id': 'req-042c2003-e8cd-47e9-a7ae-947c79b91c34'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:46.172250 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7213e8868bc4d9086c873a6fe0d49ab', 'request_id': 'req-042c2003-e8cd-47e9-a7ae-947c79b91c34'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:46.184514 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] The token cache is being invalidate because project b7213e8868bc4d9086c873a6fe0d49ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:16:46.184832 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:46.185532 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:16:46.187057 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1465/2906] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:16:45 2022] DELETE /identity/v3/projects/b7213e8868bc4d9086c873a6fe0d49ab => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:16:47.350129 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:16:47.351569 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:47.408346 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba61b7-4eb0-46c5-9a9c-814844abb81f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:16:47.457236 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad3cf8a3e0a346fa867445d6b6300447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:16:47.457993 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:16:47.458274 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:16:47.458547 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:16:47.460301 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:16:47.511595 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:47.522599 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:16:47.524015 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:16:47.534115 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1442/2907] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:16:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:16:56.355490 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:16:57.627592 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:19:13.707949 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:13.709488 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:13.921252 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c2003-e8cd-47e9-a7ae-947c79b91c34 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:13.927579 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e5b6f-32d1-42b4-a78b-c6ee076929a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '659b367f1ce643a68d55ff1c00d8b7b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:13.928218 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e5b6f-32d1-42b4-a78b-c6ee076929a4 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:13.928509 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e5b6f-32d1-42b4-a78b-c6ee076929a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:13.928812 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e5b6f-32d1-42b4-a78b-c6ee076929a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:13.930578 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-207e5b6f-32d1-42b4-a78b-c6ee076929a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:13.944132 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e5b6f-32d1-42b4-a78b-c6ee076929a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:13.945741 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e5b6f-32d1-42b4-a78b-c6ee076929a4 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:13.957438 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1466/2908] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:19:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:19:22.343292 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:22.344922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:22.495158 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2c1ba-7a34-419e-ac6e-6f0fe0da10c9 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:22.501242 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:22.501872 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:22.502161 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:22.502447 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:22.504398 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:22.516501 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:22.518078 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:22.528640 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1443/2909] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:19:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:19:23.957726 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=68 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:19:25.291998 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e842d85-bf29-4c82-aa48-a6326429d2d7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:25.298360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e842d85-bf29-4c82-aa48-a6326429d2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:25.298360 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e842d85-bf29-4c82-aa48-a6326429d2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:25.312323 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4e842d85-bf29-4c82-aa48-a6326429d2d7 None None] MFA Rules not processed for user `3001dbc46cc441cfb61b1d63a4461314`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 08:19:25.313530 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4e842d85-bf29-4c82-aa48-a6326429d2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:25.369019 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1467/2910] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 08:19:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 08:19:25.409922 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:25.411338 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:25.480023 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eeb12b-514a-45d2-a571-3a13136b63ec service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:25.485624 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '262558e0503a4f57a619c75acfdae331', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:25.486239 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:25.486519 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:25.486793 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:25.494658 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:25.496257 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:25.504717 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1444/2911] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 08:19:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 08:19:33.310719 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e842d85-bf29-4c82-aa48-a6326429d2d7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:33.323009 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e842d85-bf29-4c82-aa48-a6326429d2d7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:33.328992 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:33.329674 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:33.330021 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:33.330307 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:33.332092 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:33.344336 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:33.346099 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:33.356407 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1468/2912] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:19:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:19:33.464845 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:33.466313 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:33.480349 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:19:33.572737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d887c-3972-4217-93a9-1d49e5ac189b service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:33.572737 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:33.573247 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:33.573533 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:33.573816 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] PATH_INFO: `/v3/users/7d9d2555e71d4459b4fb907b764c1e79` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:33.575147 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d9d2555e71d4459b4fb907b764c1e79)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:33.577225 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:33.720031 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d9d2555e71d4459b4fb907b764c1e79'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:33.727392 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d9d2555e71d4459b4fb907b764c1e79'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:33.821824 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d9d2555e71d4459b4fb907b764c1e79'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:33.822427 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d9d2555e71d4459b4fb907b764c1e79'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:33.835693 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d9d2555e71d4459b4fb907b764c1e79'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:33.842070 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1445/2913] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:19:33 2022] DELETE /identity/v3/users/7d9d2555e71d4459b4fb907b764c1e79 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:19:34.074730 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:34.087074 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f595c9-0a37-4ae5-ac33-4932523ee33e service placement] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:34.148598 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:34.152083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:34.152083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:34.152083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] PATH_INFO: `/v3/projects/73e81c6e477a41ddb15e47b0c486b33e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:34.152083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e81c6e477a41ddb15e47b0c486b33e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:34.153416 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:34.270969 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e81c6e477a41ddb15e47b0c486b33e', 'request_id': 'req-24539a8c-0f96-46c1-929a-af089ce6e45e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.326316 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e81c6e477a41ddb15e47b0c486b33e', 'request_id': 'req-24539a8c-0f96-46c1-929a-af089ce6e45e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.332176 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e81c6e477a41ddb15e47b0c486b33e', 'request_id': 'req-24539a8c-0f96-46c1-929a-af089ce6e45e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.345136 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] The token cache is being invalidate because project 73e81c6e477a41ddb15e47b0c486b33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:19:34.345352 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.345989 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.347499 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1469/2914] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:19:34 2022] DELETE /identity/v3/projects/73e81c6e477a41ddb15e47b0c486b33e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:19:34.351688 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:34.353105 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:34.366467 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79c095-0518-4bd3-ba98-f4b8e2bf7c85 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:34.423049 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:34.423723 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:34.424011 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:34.424290 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] PATH_INFO: `/v3/users/00dd387613e0488cb4f9aabf9b0a409a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:34.425464 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00dd387613e0488cb4f9aabf9b0a409a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:34.427388 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:34.429666 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:19:34.557050 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00dd387613e0488cb4f9aabf9b0a409a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.565389 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00dd387613e0488cb4f9aabf9b0a409a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.632995 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00dd387613e0488cb4f9aabf9b0a409a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.633622 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00dd387613e0488cb4f9aabf9b0a409a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.646524 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00dd387613e0488cb4f9aabf9b0a409a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:34.652481 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1446/2915] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 08:19:34 2022] DELETE /identity/v3/users/00dd387613e0488cb4f9aabf9b0a409a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:19:34.889224 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:19:34.893679 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:19:34.907088 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24539a8c-0f96-46c1-929a-af089ce6e45e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:19:34.928777 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:19:34.967393 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab27936a00c428f91a5c67560723447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b4d36fd27fd480aad590e2c1f973948', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:19:34.968083 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:19:34.968369 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:19:34.968654 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] PATH_INFO: `/v3/projects/59d326c393cf4ce9b78c854f112f1b30` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:19:34.969851 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59d326c393cf4ce9b78c854f112f1b30)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:19:34.971813 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:19:35.069977 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59d326c393cf4ce9b78c854f112f1b30', 'request_id': 'req-bfbe5463-cbce-4aad-b6dd-0f425907f608'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:35.153014 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59d326c393cf4ce9b78c854f112f1b30', 'request_id': 'req-bfbe5463-cbce-4aad-b6dd-0f425907f608'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:35.159295 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59d326c393cf4ce9b78c854f112f1b30', 'request_id': 'req-bfbe5463-cbce-4aad-b6dd-0f425907f608'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:35.171248 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] The token cache is being invalidate because project 59d326c393cf4ce9b78c854f112f1b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 08:19:35.171562 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:35.172221 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bfbe5463-cbce-4aad-b6dd-0f425907f608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 08:19:35.173742 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1470/2916] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 08:19:34 2022] DELETE /identity/v3/projects/59d326c393cf4ce9b78c854f112f1b30 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 08:19:44.648894 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:19:45.167473 dv-cinder-740146-16-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 08:24:31.586079 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 08:24:31.587740 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:24:31.658558 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9b2a4-36cd-4da9-9694-e068f5344224 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 08:24:31.718912 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b173a1-c9ad-4bfc-ae04-2885b8503371 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b26619beabb4e8f96d9f08ecb107027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b176962cc74ab88204a4271aad8278', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 08:24:31.719553 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b173a1-c9ad-4bfc-ae04-2885b8503371 service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 08:24:31.719919 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b173a1-c9ad-4bfc-ae04-2885b8503371 service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 08:24:31.720196 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b173a1-c9ad-4bfc-ae04-2885b8503371 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 08:24:31.721962 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-33b173a1-c9ad-4bfc-ae04-2885b8503371 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 08:24:31.733228 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b173a1-c9ad-4bfc-ae04-2885b8503371 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 08:24:31.734806 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b173a1-c9ad-4bfc-ae04-2885b8503371 service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 08:24:31.745089 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1447/2917] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 08:24:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 08:24:41.740405 dv-cinder-740146-16-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=58 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}